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EXECUTIVE SUMMARY 


This report describes the test results of the runway incursion alerting systems recorded during the 
NASA Runway Incursion Prevention System (RIPS) testing at Dallas - Fort Worth International 
Airport (DFW) in October 2000. Both aircraft-based and ground-based runway incursion 
alerting were implemented and tested. The Runway Safety Monitor (RSM) and Runway 
Incursion Advisory and Alerting System (RIAAS) are aircraft-based runway incursion alerting 
systems. RSM was developed in-house by NASA. RIAAS was developed by Rannoch 
Coiporation. The John A. Volpe National Transportation Systems Center (VTNSC) 
implemented the Ground-Based System (GBS). Prototype versions of RIAAS and RSM were 
installed on NASA’s B757 aircraft (also called the Airborne Research Integrated Experiments 
System or ARIES). 

The objectives of the RIPS flight test were [Ref. 1]: 

Assess and validate the performance of Communications Navigation and Surveillance (CNS) 
infrastructure technologies and incursion alerting systems for preventing runway incursion 
accidents. Specific objectives were; 

• Assess the performance of the airport surface infrastructure (data linked Surface Traffic 
Information Service - Broadcast (STIS-B) with runway incursion alerting) for providing 
sufficient situational awareness and warning to prevent runway incursion accidents. 

• Assess the performance of aircraft-based runway incursion alerting systems for providing 
sufficient situational awareness and warning to prevent runway incursion accidents 
utilizing the following data sources: 

(a) STIS-B from airport surface infrastructure 

(b) Automatic Dependent Surveillance - Broadcast ( ADS-B ) aircraft to aircraft data link 

Runway incursion scenarios were performed using the B757 and a ground vehicle. Some of the 
key performance measures analyzed include warning response times, missed detection 
performance, false alert generation, and surveillance latency. 

The primary conclusion of this report is that the three types of approaches to generating runway 
incursion alerts in the cockpit demonstrated feasibility during the DFW RIPS testing. Out of the 
47 test runs, RIAAS provided alerts on 44, RSM on 43, and GBS on 34. All of the missed alerts 
on RIAAS and RSM were a direct result of erroneous or missing traffic data. Most of the missed 
alerts for GBS were related to the original alerting criteria, which were changed part way through 
testing. Other missed GBS alerts were mostly due to the design of some specific scenarios where 
the GBS alerting criteria were not satisfied. In these instances the relative locations of the 
aircraft and test vehicle did not meet the GBS criteria for alert. RIAAS generated 2 false alerts 
during the testing, both the result of erroneous traffic data. RSM generated 4 false alerts, which 
were the result of the ownship-generated STIS-B traffic reports. GBS generated 9 false alerts 
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during the testing, most of which were due to an apparent false ASDE-3 target located off the 
runway. 

The testing showed that the pilot could safely take evasive action (i.e. go-around, rejected take 
off, stop taxi) when the alerts normally occurred on all three systems for the four incursion 
scenarios tested. However, for the scenarios involving violation of hold lines, the GBS alerts 
occurred significantly later than for the aircraft-based systems. In those two scenarios (1 and 3) 
the GBS alerts did not occur until the vehicle/aircraft was on the runway. The two aircraft-based 
systems alerted well before the vehicle and aircraft reached the runway. 

Regarding the integration of the supporting airborne and ground systems, the test results indicate 
that the basic system architecture demonstrated at DFW will support both aircraft-based and 
ground-based incursion alerting. One conclusion, as expected, is that alert logic performance is 
very dependent on the performance of the traffic and ownship position information. This 
information must be reliable, timely and accurate to ensure optimum runway incursion alerting 
performance. The NASA B757 airborne systems demonstrated excellent performance with 
respect to ownship information. However, there were a number of issues identified regarding the 
generation and processing of traffic information using STIS-B and ADS-B. Missing or erroneous 
STIS-B and ADS-B data resulted in a number of missed, late, and false alerts. The prototype 
nature of the systems involved is believed to have played a significant role in the availability and 
integrity of the traffic data. One specific conclusion with regard to traffic information is that 
STIS-B information had significantly longer latency than did ADS-B. This translates directly 
into delayed alerting on targets using position reports from STIS-B. ADS-B position reports 
were also significantly more accurate than STIS-B. 

RIAAS demonstrated a two-stage alerting concept, which includes a Traffic Alert and a higher 
priority Conflict Alert. The other two systems, RSM and GBS, provided a single conflict alert. 
The intent of the two stage alerting is to provide advanced warning to the pilot of a pending 
conflict. For most of the scenarios tested, the RIAAS two stage alerting worked as designed, 
providing time between the two alerts (as much as 10-20 seconds) for the pilot to determine the 
best course of action. Further simulation and testing is required to validate and optimize the two- 
stage alerting approach. 

The testing demonstrated that aircraft-based alerting has several key advantages over ground- 
based alerts provided via data link, including: 

• Shorter time delay between alert generation and annunciation of alerts to the flight crew. 

• More timely alert generation. One reason for this is the capability to use ownship 
position data to accurately determine the ownship nose location. This provides a means 
to very accurately determine when ownship has violated a hold line on entering a runway. 
A similar computation can be made for the tail location to determine when an aircraft has 
failed to clear the hold line on exiting a runway. 

• Ground infrastructure is not required when aircraft are equipped with ADS-B. 



Aircraft-based alerts provided to the flight crew will in some cases occur in advance of ground- 
based alerts provided to ATC. For example, in the case where ownship violates the hold line, an 
aircraft-based alert can occur sooner than the ground-based alerts due to the ability to accurately 
determine nose position. There is a safety benefit to alerting the flight crew as soon as the 
aircraft has crossed the hold line. This may present an issue regarding the difference in timing 
for the two alerting systems. The compatibility of aircraft-based alerts reported to the flight crew 
and ground-based alerts reported to ATC needs further investigation. 

Analysis of the test results yielded several recommendations regarding the supporting 
infrastructure and the alerting systems, including: 

• Further development of ground and avionics systems should include enhancement of 
availability and integrity of ADS-B and STIS-B traffic infoimation. The ground system 
should provide integrity monitoring of surveillance data prior to STIS-B transmission. 
STIS-B should transmit a parameter equivalent to the ADS-B Navigation Uncertainty 
(NUC). This will indicate the accuracy of the surveillance information. The latency in 
the STIS-B transmissions should also be minimized to reduce alert delays. 

• A reference point correction for the ADS-B target should be performed. It is 
recommended that the ADS-B MASPS be amended to include a requirement that the 
reported position is referenced to a standard location on the aircraft. If the position is 
provided to a known location then the alerting systems can apply the correction to other 
critical aircraft points of reference (i.e., nose, tail). 

• The ground system should provide STIS-B position reports that are corrected to a 
reference point, such as the nose or centroid of the aircraft. The ground system has 
knowledge of the surveillance sensor(s) used to determine the fused position. Each 
sensor can use a different reference point. For instance, ASDE-3 position is referenced to 
the target centroid and multilateration position is referenced to the transponder antenna(s) 
location. The avionics does not have the knowledge of which sensor is used to compute 
the ground system derived traffic reports. 

• Aircraft-based incursion alerting systems should incoiporate some level of integrity 
checking on traffic information to minimize missed and false alerts. 



Table of Contents 


EXECUTIVE SUMMARY i 

ACKNOWLEGEMENTS vii 

1.0 INTRODUCTION 1 

2.0 RIPS SYSTEM DESCRIPTION 2 

2.1 Avionics Systems Architecture 2 

2.1.1 RIAAS Aircraft-Based Alerting 4 

2. 1 .2 RSM Aircraft-Based Alerting 6 

2.1.3 Ground-Based Alerting 7 

2.2 RIPS Ground-Based System Architecture 8 

2.2.1 Multilateration 10 

2.2.2 ASDE-3 Radar 10 

2.2.3 Surveillance Server 1 1 

2.2.4 Automatic Dependent Surveillance Broadcast (ADS-B) 1 1 

2.2.5 STIS-B/FIS-B Data Link 1 1 

3.0 DATA COLLECTION AND ANALYSIS METHODOLOGY 12 

3.1 Test Scenarios 12 

3.2 Data Collection 15 

4.0 TEST ANALYSIS AND RESULTS 19 

4.1 STIS-B Surveillance Evaluation 19 

4.1.1 Latency 19 

4.1.2 Update Rate 19 

4. 1 .3 Position Accuracy 20 

4.1.4 Speed Accuracy 20 

4.1.5 Track Angle Accuracy 22 

4.1.6 Data Integrity 23 

4.2 ADS-B Surveillance Evaluation 23 

4.2.1 Latency 23 

4.2.2 Update Rate 23 

4.2.3 Position Accuracy 24 

4.2.4 Speed Accuracy 24 

4.2.5 Track Angle Accuracy 24 

4.2.6 Data Integrity 27 

4.3 Ownship Navigation Evaluation 27 

4.3.1 LAAS Accuracy 27 

4.3.2 Conclusions 27 

4.4 Runway Incursion Alerting Evaluation 27 

4.4. 1 RIAAS Alerting Performance 27 

4.4. 1 . 1 RIAAS Scenario Alert Plots 27 

4.4. 1.2 RIAAS Alert Performance Summary 28 

4.4.2 RSM Alerting Performance 41 

4.4.2. 1 RSM Scenario Alert Plots 41 

4. 4. 2. 2 RSM Alert Performance Summary 42 

4.4.3 Ground-Based System Alerting Performance 48 


IV 



4.4.3. 1 GBS Scenario Alert Plots 48 

4. 4. 3. 2 GBS Alert Performance Summary 48 

4.4.4 Relative Comparison of Alert Performance 55 

4.4.4. 1 Combined Alert Plots 55 

4. 4. 4. 2 Combined Alert Performance Summary 56 

5.0 SUMMARY AND CONCLUSIONS 59 

REFERENCES 62 


APPENDIX A -RIAAS ALERT PLOTS 
APPENDIX B - RIAAS ALERT SUMMARY 
APPENDIX C- RSM ALERT PLOTS 
APPENDIX D - RSM ALERT SUMMARY 
APPENDIX E- GBS ALERT PLOTS 
APPENDIX F- GBS ALERT SUMMARY 
APPENDIX G- COMBINED ALERT PLOTS 
APPENDIX H - FALSE ALERTS 
APPENDIX I - ACRONYMS 


Figures 


Figure 1. Avionics Systems Architecture 3 

Figure 2. Alert Displays in ARIES B 75 7 4 

Figure 3. Runway Safety Monitor State Matrix 7 

Figure 4. RIPS Ground System Architecture 10 

Figure 5. RIPS Scenario 1 13 

Figure 6. RIPS Scenario 2 14 

Figure 7. RIPS Scenario 3 16 

Figure 8. RIPS Scenario 4 1 7 

Figure 9. STIS-B Reception Performance 20 

Figure 10A. Traffic STIS-B Position Accuracy Compared to ADS-B 21 

Figure 1 OB. Owns hip STIS-B Position Accuracy 21 

Figure 11. STIS-B Position Along Track Accuracy (NASA B757) 22 

Figure 12. STIS-B Track Angle Plot (NASA B757) 22 

Figure 13. ADS-B Reception Performance of Test Vehicle Transmissions 24 

Figure 14. ADS-B vs. STIS-B Track Angle for Test Vehicle 25 

Figure 15. ADS-B versus STIS-B Speed for Test vehicle 25 

Figure 16. ADS-B Position for Test Vehicle Traveling on Centerline 26 


V 



Tables 


Table 1. RIAAS Alert State Pairs 5 

Table 2. GBS Alert Situations for Two Target Alerts 9 

Table 3. Experiment Matrix (NASA B7 57) 18 

Table 4. Scenario 1 RIAAS Alert Summary 32 

Table 5. Scenario 2 RIAAS Alert Summary 33 

Table 6. Scenario 3 RIAAS Alert Summary 33 

Table 7. Scenario 4 RIAAS Alert Summary 34 

Table 8. Scenario 1 RIAAS RTAs 34 

Table 9. Scenario 1 RIAAS RCAs 35 

Table 10. Scenario 1 Time Between RIAAS Alerts 35 

Table 11. Scenario 2 RIAAS RTAs 36 

Table 12. Scenario 2 RIAAS RCAs 36 

Table 13. Scenario 2 Time Between RIAAS Alerts 37 

Table 14. Scenario 3 RIAAS RTAs 37 

Table 15. Scenario 3 RIAAS RCAs 38 

Table 16. Scenario 3 Time Between RIAAS Alerts 38 

Table 17. Scenario 4 RIAAS RTAs 39 

Table 18. Scenario 4 RIAAS RCAs 39 

Table 19. Scenario 4 Time Between RIAAS Alerts, 40 

Table 20. Scenario 1 RSM Alert Summary, 44 

Table 21. Scenario 2 RSM Alert Summary 44 

Table 22. Scenario 3 RSM Alert Summary, 45 

Table 23. Scenario 4 RSM Alert Summary 45 

Table 24. Scenario 1 RSM Alerts, 46 

Table 25. Scenario 2 RSM Alerts 46 

Table 26. Scenario 3 RSM Alerts 47 

Table 27. Scenario 4 RSM Alerts 47 

Table 28. Scenario 1 GBS Alert Summary 51 

Table 29. Scenario 2 GBS Alert Summary 51 

Table 30. Scenario 3 GBS Alert Summary 52 

Table 31. Scenario 4 GBS Alert Summary 52 

Table 32. Scenario 1 GBS Alerts 53 

Table 33. Scenario 2 GBS Alerts 53 

Table 34. Scenario 3 GBS Alerts 54 

Table 35. Scenario 4 GBS Alerts 54 

Table 36. RIAAS Alert Summary By Data Source 58 

Table 37. RSM Alert Summary By Data Source 58 

Table 38. GBS Alert Summary > By Data Source 58 


vi 



ACKNOWLEGEMENTS 


The authors gratefully acknowledge the contributions of the NASA Langley Runway Incursion 
Prevention System team, especially Denise Jones, for their guidance and assistance in the 
research presented in this report. We would also like to acknowledge the assistance provided by 
David Green of Lockheed Martin and Sharon Woods of the Volpe National Transportation 
Systems Center. 



1.0 INTRODUCTION 

Airport surface incursions have been identified as one of the most significant safety hazards in 
civil aviation [1], The Runway Incursion Prevention System (RIPS) is being developed by 
NASA in cooperation with the Federal Aviation Administration (FAA) to help address this 
problem. RIPS builds on the airport ground systems infrastructure put in place by the FAA. 
This ground infrastructure includes the following functions: 

• Surface Surveillance - Provides surveillance of airport surface traffic. 

• Ground-Based System (GBS) Alerting - Processes the traffic information to identify 
runway incursions and provide alerts for presentation to the Air Traffic Controllers or the 
flight crews. 

• Surface Traffic Information Services - Broadcast (STIS-B) - Transmits surface 
surveillance traffic information to aircraft and ground vehicles. 

• Local Area Augmentation System (LAAS) - Provides aircraft and ground vehicles with 
differential corrections to GPS navigation. 

• Controller pilot data link - Provides ATC taxi instructions to the flight crew via a data 
link. 

RIPS avionics supports enhanced safety by providing the flight crews with information regarding 
navigation, traffic movement, and runway incursions. This information is available on a Cockpit 
Display of Traffic Information (CDTI). Additionally, a Heads Up Display (HUD) provides 
navigation and traffic information directly to the pilot. Audible alerting is provided in 
conjunction with the display devices. Traffic information is obtained from the ground 
infrastructure via STIS-B, and can also be obtained directly from other aircraft via Automatic 
Dependent Surveillance - Broadcast (ADS-B). 

This report describes the performance results of the runway incursion alerting systems recorded 
during the NASA Runway Incursion Prevention System (RIPS) testing at Dallas - Fort Worth 
International Airport (DFW) in October 2000. Both aircraft-based and ground-based runway 
incursion alerting systems were implemented and tested. Runway Safety Monitor (RSM) and 
Runway Incursion Advisory and Alerting System (RIAAS) are aircraft-based runway incursion 
alerting systems. RSM was developed in-house by NASA. RIAAS was developed by Rannoch 
Corporation. The John A. Volpe National Transportation Systems Center (VTNSC) developed 
the GBS. Prototype versions of RIAAS and RSM were installed on NASA’s B757 aircraft, the 
Airborne Research Integrated Experiments System (ARIES). 

The RIAAS and RSM systems provide runway incursion alerts directly to the flight crews. 
While the FAA is in the process of implementing ground-based alerting for Air Traffic Control 
(ATC) tower controllers, there is no operational system to alert pilots automatically at the onset 
of such conflicts. Ground-based alerts must be relayed by ATC, via voice communications, to 
the flight crew. The flight crew does not have the same level of situational awareness as ATC, 
because they lack the situational display of traffic information. The time delay associated with 
alert communication, combined with the lack of traffic information in the cockpit, limits the 
effectiveness of ground-based alerting implementations. Aircraft-based alerting can help 
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minimize the risk of a runway incursion, in certain scenarios, through advanced traffic alerting 
prior to the occurrence of a runway incursion. In the event that a scenario develops into a runway 
incursion, conflict alerting provides the flight crew with timely information so that evasive action 
can be taken. 

The objectives of the RIPS flight test were [Ref. 1]: 

Assess and validate performance of Communications Navigation and Surveillance (CNS) 
infrastructure technologies and incursion alerting systems for preventing runway incursion 
accidents. Specific objectives were; 

• Assess the performance of the airport surface infrastructure (data linked STIS-B with 
runway incursion alerting) for providing sufficient situational awareness and warning to 
prevent runway incursion accidents. 

• Assess the performance of aircraft-based runway incursion alerting systems, utilizing 
STIS-B traffic data provided by the airport surface infrastructure and data provided by an 
ADS-B aircraft to aircraft data link, in providing sufficient situational awareness and 
warning to prevent runway incursion accidents. 

Runway incursion scenarios were performed using the B757 and a ground vehicle. Some of the 
key performance measures analyzed include warning response times, missed detection 
performance, false alert generation and surveillance latency. 

2.0 RIPS SYSTEM DESCRIPTION 

The NASA Runway Incursion Prevention System consists of both avionics and ground systems 
elements [3], The ground elements provide traffic information to the avionics elements. The 
avionics elements process this traffic information to provide runway incursion alerting. The 
avionics elements are also designed to support runway incursion alerting in the absence of the 
ground system elements using aircraft-to-aircraft surveillance provided by ADS-B. 

2.1 Avionics Systems Architecture 

Figure 1 shows the system architecture for the avionics installed on the B757 to support RIPS. A 
SGI Onyx served as the hardware platform for the RIAAS and RSM software. Runway incursion 
alerts were displayed on a HUD, a Navigation Display (ND), and an Electronic Moving Map 
(EMM), illustrated in Figure 2. A raster-style HUD supporting resolutions of up to 1280x959 
was used. The HUD displayed the alert type and the distance and time to conflict in the event of 
an incursion alert. The EMM presented an ownship proximate view of the movement area and 
traffic information. Intruding traffic and its location were identified by highlighting that traffic’s 
symbol. The color of the highlighted traffic symbol indicated the type of alert, yellow for 
Runway Traffic Alerts and red for Runway Conflict Alerts. The RIPS Audio Alert System was 
used to provide runway incursion alert annunciations in the cockpit. The system was comprised 
of a digital audio recorder/player and a speaker (part of the ARIES audio system). Runway 
Traffic Alerts were annunciated in the cockpit as “Runway Traffic, Runway Traffic.” Runway 
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Conflict Alerts were annunciated as “Runway Conflict, Runway Conflict.” Textual forms of 
these messages were also displayed on the HUD and EMM. 

Ownship position was provided by LAAS differentially corrected Global Positioning System 
(GPS) data and the Inertial Navigation System (INS) data. An INS/GPS blending technique was 
implemented to enhance position accuracy. This process involves filtering the DGPS 
(Differential GPS) position with the INS position to produce a blended solution. 

Traffic information is obtained from both the 1090 MHz ADS-B and a STIS-B data link. A 
Universal Access Transceiver (UAT) data link was used to provide STIS-B data to the B757. 



Figure 1. Avionics Systems Architecture 
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HUD Guidance 


Electronic Moving Map 


Figure 2. Alert Displays in ARIES B757 

2.1.1 Rl AAS Aircraft-Based Alerting 

RIAAS is designed to monitor aircraft that are either on the airport surface, or are within the 
airport’s arrival and departure zones [Ref. 2], RIAAS initiates alert processing whenever the 
aircraft on which it is installed (ownship) enters a runway zone, which includes the runway, 
intersecting taxiways, arrival and departure zones associated with the runway. The system uses 
ADS-B and/or STIS-B to track other aircraft or ground vehicles (traffic) operating in ownship’s 
runway zone. RIAAS is configured to issue alerts based on the states and proximity of traffic 
relative to ownship. 

RIAAS is an aircraft-based safety alerting system designed to identify early conditions for 
runway incursions and provide aircraft pilots and ground vehicle operators sufficient time to 
avoid runway incursion conflicts and collisions when an alert is issued. The alerting logic is the 
core of the RIAAS algorithms. RIAAS also requires a method for annunciating the alerts. Alerts 
may be annunciated aurally and/or visually. A typical implementation would be to display alerts 
on a CDTI and provide aural alerts to draw the flight crew’s attention to the incursion situation. 

RIAAS is designed to handle over forty different runway incursion scenarios, as listed in Table 1. 
Parameters such as position, speed, acceleration, heading, distance to hold lines, distance to 
thresholds, distance to runway edge, closure rate and separation distance are measured for every 
vehicle operating in the vicinity of the runway being used. Calculations of each vehicle’s 
dynamic state are compared against the alerting criteria, and an alert is issued if the criteria are 
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Table 1. RIAAS Alert State Pairs 


Scenario 

Ownship 

Other Vehicle 


Pair 

State 

State 

Conflict 

1 

Arrival 

Taxi 

Crossing 

2 

Arrival 

Taxi 

Tail Chase 

3 

Arrival 

Taxi 

Tail Lead 

4 

Arrival 

Taxi 

Head On 

5 

Taxi 

Arrival 

Crossing 

6 

Taxi 

Arrival 

Tail Chase 

7 

Taxi 

Arrival 

Tail Lead 

8 

Taxi 

Arrival 

Head On 

9 

Departure 

Taxi 

Crossing 

10 

Departure 

Taxi 

Tail Chase 

11 

Departure 

Taxi 

Tail Lead 

12 

Departure 

Taxi 

Head On 

13 

Taxi 

Departure 

Crossing 

14 

Taxi 

Departure 

Tail Chase 

15 

Taxi 

Departure 

Tail Lead 

16 

Taxi 

Departure 

Head On 

17 

Arrival 

Departure 

Crossing 


18 

Arrival 

Departure 

Tail Chase 

19 

Arrival 

Departure 

Tail Lead 

20 

Arrival 

Departure 

Head On 

21 

Departure 

Arrival 

Crossing 

22 

Departure 

Arrival 

Tail Chase 

23 

Departure 

Arrival 

Tail Lead 

24 

Departure 

Arrival 

Head On 

25 

Arrival 

Arrival 

Crossing 

26 

Arrival 

Arrival 

Tail Chase 

27 

Arrival 

Arrival 

Tail Lead 

28 

Arrival 

Arrival 

Head On 

29 

Departure 

Departure 

Crossing 

30 

Departure 

Departure 

Tail Chase 

31 

Departure 

Departure 

Tail Lead 

32 

Departure 

Departure 

Head On 

33 

Taxi 

Taxi 

Crossing 

34 

Taxi 

Taxi 

Tail Chase 

35 

Taxi 

Taxi 

Tail Lead 

36 

Taxi 

Taxi 

Head On 

37 

Arrival 

Stopped 

Head On 

38 

Departure 

Stopped 

Head On 

39 

Taxi 

Stopped 

Head On 

40 

Taxi 

Stopped 

Crossing 

41 


Arrival 

Head On 

42 


Arrival 

Tail Lead 

43 

Stopped 

Departure 

Head On 

44 

Stopped 

Departure 

Tail Lead 

45 

Stopped 

Taxi 

Head On 

46 

Stopped 

Taxi 

Crossing 


















































































































































































met for one or more incursion scenarios. If multiple alert scenarios occur simultaneously, the 
one with the highest level of alert is used in determining which alert will be issued. Once 
corrective action has been taken and there is no longer a state of alert, the alerts are cleared from 
the display. 

RIAAS provides two stages of alerting, analogous to TCAS. A Runway Traffic Alert (RTA) is 
generated when own aircraft is either projected to be involved in a runway incursion with other 
traffic or an incursion has occurred that does not yet require evasive action. A Runway Conflict 
Alert (RCA) is provided when an actual runway incursion has been detected, and there is 
potential for collision. An RCA indicates that the aircraft involved in the conflict needs to take 
evasive action to avoid the potential collision. RIAAS, as well as the other alerting systems, does 
not provide guidance information to the pilot for taking evasive action. The reason for this is that 
the number and complexity of the potential scenarios makes it difficult to correctly identify the 
proper evasive action to take in every situation. Information that is provided with each alert 
includes identification of the incurring aircraft (or vehicle), the runway associated with the 
aircraft, separation distance and time to conflict. Alerts can be displayed on a moving map 
display tailored to the airport surface. This display should provide enough information to the 
pilot to determine proper evasive action. 

2.1.2 RSM Aircraft-Based Alerting 

Runway Safety Monitor ( RSM) is an alerting element provided by the Integrated Display System 
(IDS), a NASA developed experimental avionics display and data communications system for 
landing and surface operations [Ref. 3]. RSM is a single stage alerting system that provides 
Runway Conflict Alerts (RCAs). The system uses either STIS-B or ADS-B data as the source for 
traffic information. Selection of traffic source is done manually. The system uses a generic 
approach, which requires information on the location of the runways, but does not use 
information on the location of the taxiway hold lines. Runway incursion zones are monitored 
and established as follows: 

Sides of zone: 220 feet from edge of runway 
Ends of zone: 1 . 1 nm from runway threshold 
Altitude of zone: 400 feet above airport surface 

On initialization, RSM reads a configuration file and computes/stores coordinates for all runway 
incursion zones. The logic does not address taxi-only operations where both ownship and traffic 
are considered to be in a taxi state. The algorithm is divided into three main parts: 

Part 1 : Invoking RSM and determining when to start/stop/continue incursion monitoring 
(ownship inside any incursion zone?) 

Part 2 : Identifying and tracking all targets inside current runway incursion zone. 
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Part 3 : Determining target/ownship states, testing for incursion alert conditions (Figure 3 
State matrix) and setting or clearing alert data. 


Check Operational State Conditions: Incursion YES/NO 


Target Taxi 
Or Not Moving 

Target 
Land, T/O 

Target 

Fly-thru 

Owns hip Taxi 
Or Not Moving 

NO 

YES 
if closing 

NO 

Ownship 
Land, T/O 

YES 
if closing 

YES 

if < min separation 

YES 

if closing AND 
< min separation 

Ownship 

Fly-thru 

NO 

YES 

if closing AND 
< min separation 

NO 

No Incursions 
Clear Alert Flags/Data 

TNCl IRSIONis) Detected: 
Sot Alert Piags/Oata 


Figure 3. Runway Safety Monitor State Matrix 


2.1.3 Ground-Based Alerting 

The GBS safety logic utilized at DFW was a subset of the Airport Movement Area Safety System 
(AMASS) alerting logic used in the operational systems [Ref. 4], The GBS is resident on the 
Surveillance Server. GBS analyzes traffic location and movement to identify runway incursion 
situations and other potential hazards. The GBS also provides generation of hold bar indications, 
which are transmitted and can be viewed on the cockpit display to indicate when it is unsafe to 
enter the runway. The GBS receives tracks three times per second from the Fusion Process of the 
Surveillance Server. Tracks are maintained in a database. At an interval of 1 second, this 
database is analyzed for potential hazards. When alerts or hold bars are generated, the 
information is sent to the aircraft via STIS-B. Safety Logic confines its analysis of alert 
situations to the active runways of the airport. Tracks are placed in a list of tracks associated 
with each runway. Tracks not on a runway or in an approach window are not processed. Tracks 
in each runway list are then assigned movement states (i.e., ARRIVAL, LANDING, STOP, 
TAXI, DEPARTURE, DEPARTURE_ABORT). For each of the runways, safety logic checks 
each track first to determine whether it qualifies as a one-track alert, and then compares it to all 
the other tracks in the runway list for the possibility of a two-track alert. One-track alerts include 
Arrival on a Closed Runway and Stop-Timeout. If a track is in the ARRIVAL state and is 
assigned to a closed runway, a Closed Runway alert is generated. A Stop-Timeout alert is 
generated when a target is in the STOP state on an active runway for a period that exceeds a user 
selectable timeout period. For analysis of two-track alerts, each track in a runway’s list is 


7 



compared to all the other tracks in the same list. Table 2 lists two-target alert situations for GBS. 

The Safety process is as follows: 

1) Determine direction of each track as normal or opposite to the runway’s designated 
direction. 

2) Designate one track as Track A and the other as Track B. 

3) Compute the separation distance between the two tracks. 

4) Match the two tracks to an alert situation as identified in Table 2. (Note, dir = direction of 
movement, Mvmt. State = Movement State, N = Normal direction, O = Opposite 
direction) 

2.2 RIPS Ground-Based System Architecture 

The RIPS ground architecture, illustrated in Figure 4, includes the following elements: 

1. Airport Surface Detection Equipment (ASDE-3) radar - Provides surveillance (position 
only) of aircraft or vehicles operating on the runway/taxiway area. 

2. Airport Surface Target Identification System (ATIDS) - Provides surveillance (position 
and ID) of aircraft and ground vehicles equipped with 1090 MHz ADS-B, Mode-S 
transponders, and Mode A/C transponders. 

3. Surveillance server - Provides the following: 

a) Tracking of ASDE-3 targets 

b) Data fusion of ATIDS target data with ASDE-3 track data to enhance situational 
awareness for Air Traffic Control (ATC) and flight crews with Cockpit Display of 
Traffic Information (CDTI) 

c) Ground-based Alerting safety logic to detect runway incursions and other conflicts. 

4. Local Area Augmentation System (LAAS) DGPS ground station - Provides differential 
corrections for navigation and surveillance. 

5. Surface Traffic Information Services - Broadcast (STIS-B )/ Flight Information Services - 
Broadcast (FIS-B) digital data link system - Provides the following to data link equipped 
aircraft: 

a) Digital transmission of traffic information 

b) Runway hold bar information 

c) Ground generated alerts. 

6. Automated Radar Tracking System (ARTS) - Provides ASR-9 radar position/ID of 
airborne aircraft near the airport. 



Table 2. GBS Alert Situations for Two Target Alerts 


SITUATION 


TRACK A 

TRACK B 

ALERT INFORMATION 

dir 

Mvmt. State 

dir 

Mvmt. State 


N 

DEPARTURE 

O/N 

STOP 

(Track A ID) AND (Track B ID), (Runway ID), 
DEP, OCCUPIED RUNWAY 

O 

DEPARTURE 

O/N 

STOP 

(Track A ID) AND (Track B ID), (Runway ID), 
OPPOSITE DIRECTION DEP 

N 

LANDING 

O/N 

STOP 

(Track A ID) AND (Track B ID), (Runway ID), 
LDG, OCCUPIED RWY 

O 

LANDING 

O/N 

STOP 

(Track A ID) AND (Track B ID), (Runway ID), 
OPPOSITE DIRECTION LDG, OCCUPIED RWY 

N 

DEPARTURE 

O 

DEPARTURE 

(Track A ID) AND (Track B ID), (Runway ID), 





HEAD-ON DEPS 

N 

DEPARTURE 

o 

LANDING 

(Track A ID) AND (Track B ID), (Runway ID), 
HEAD-ON TRAFFIC 

N 

LANDING 

o 

DEPARTURE 

(Track A ID) AND (Track B ID), (Runway ID), 
HEAD-ON TRAFFIC 

N 

ARRIVAL 

N 

DEPARTURE 

(Track A ID) AND (Track B ID), (Runway ID), 
ARR, OCCUPIED RWY 

N 

ARRIVAL 

o 

DEPARTURE 

(Track A ID) AND (Track B ID), (Runway ID), 
HEAD-ON TRAFFIC 

N 

ARRIVAL 

o 

LANDING 

(Track A ID) AND (Track B ID), (Runway ID), 
HEAD-ON TRAFFIC 

N 

ARRIVAL 

N 

TAXI 

(Track A ID) AND (Track B ID), (Runway ID), 
ARR, OCCUPIED RWY 

N 

ARRIVAL 

o 

TAXI 

(Track A ID) AND (Track B ID), (Runway ID), 
HEAD-ON TRAFFIC 

N 

ARRIVAL 

O/N 

STOP 

(Track A ID) AND (Track B ID), (Runway ID), 
ARR, OCCUPIED RWY 
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2.2.1 Multilateration 


Multilateration and target identification was accomplished with an ATIDS system. ATIDS is 
based on Secondary Surveillance Radar (SSR) technology and is an enhancement to current 
airport primary surveillance equipment, which at DFW is ASDE-3/Surveihance Server. ATIDS 
augments the ASDE-3/Surveillance Server surveillance with aircraft identification and 
surveillance to fill in coverage gaps of the ASDE-3 radar. ATIDS is a multilateration system that 
receives SSR transmissions from aircraft and triangulates, or multilaterates, from several receiver 
locations to pinpoint the location of an SSR transponder. The system is designed to operate in 
conjunction with aircraft equipped with Mode A/C and Mode S transponders. 


Traffic Information 
& Alerts 



Figure 4. RIPS Ground System Architecture 


2.2.2 ASDE-3 Radar 

The ASDE-3 is a Ku band primary radar used for airport movement area surveillance. It is 
intended to provide controllers with enhanced visibility of airport surface traffic in low visibility 
conditions, thereby increasing safety and reducing runway incursions. It uses an antenna rotating 
once per second, resulting in a target update at the same rate. The ASDE-3 provides surveillance 
of aircraft and vehicles operating on runways and taxiways that are in direct line of site to the 
radar. Non-movement areas such as grass and ramp areas are intentionally filtered out. The 
ASDE-3 installed at DFW is a commissioned production unit installed on top of the air traffic 
control tower. 
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2.2.3 Surveillance Server 


The Surveillance Server is a prototype system that takes radar return inputs from the ASDE-3 
and digitizes them. It then determines the centroid and extent information of the airport surface 
targets. The Surveillance Server fuses data from the following sources: 

• ARTS arrival database information 

• ASDE-3/Surveillance Server target track information 

• ATIDS 1090 MHz ADS-B target information, and 

• ATIDS 1090 MHz multilateration target information 

The resulting fused surveillance data is output to a controller interface and to a datalink manager 
to be transmitted to the NASA B757 via STIS-B. Using this digitized data, the Surveillance 
Server can track aircraft and vehicles on the airport surface and provide automatic warnings of 
conflicts and runway incursions. 

2.2.4 Automatic Dependent Surveillance Broadcast (ADS-B) 

ADS-B is a function on an aircraft that periodically broadcasts the aircraft state vector (position 
and velocity) [4], Air traffic control can receive the state vector reports to accurately display 
traffic identity and position. Other aircraft can receive the information for use in collision 
avoidance and CDTI applications. 

ADS-B, as implemented in the RIPS tests, consisted of a Collins GPS receiver and Mode S 
extended squitter transponder installed on the NASA B757 as well as on a ground vehicle. 
Differential GPS corrections were obtained from LAAS. Position was calculated 5 times per 
second and the most recently computed position was transmitted nominally twice per second. 
Two different ADS-B messages were transmitted, depending on whether the aircraft was 
airborne or on the airport surface. The airborne ADS-B message includes type code (information 
on airborne or surface message and precision category of the data), surveillance status, turn 
indicator (turning or not turning), altitude (either barometric or GNSS derived), and encoded 
latitude and longitude (17 bits). The surface ADS-B message includes type code (same as 
airborne), ground speed, track angle and encoded latitude and longitude. ADS-B transmissions 
alternate between the top and bottom mount antennas when airborne. ADS-B transmissions are 
only radiated from a top mount antenna when the aircraft is on the ground. In the RIPS test 
vehicle at DFW two antennas were used to broadcast transmissions, alternating at half second 
intervals. 

2.2.5 STIS-B/FIS-B Data Link 

STIS-B and FIS-B are uplinked to the B757 via a UAT data link. Traffic information is updated 
once per second. FIS-B transmitted data includes ground-based alerts and hold bar indications. 
UAT operates in the L band and accordingly requires line-of-sight between the ground-based and 
aircraft-based transceivers. 


11 



3.0 DATA COLLECTION AND ANALYSIS METHODOLOGY 


3.1 Test Scenarios 

A complete description of the tests can be found in the NASA test plan [Ref. 1], All RIPS 
testing was performed at Dallas - Fort Worth International Airport, at night, and under good 
visibility conditions. Four RIPS scenarios were tested as illustrated in Figures 5 through 8. In 
each scenario, the NASA B757 was involved in an incursion with a ground test vehicle. 

RIPS Scenario 1 - Arrival (NASA B757VTaxi (Test vehicle) 

Scenario 1 is illustrated in Figure 5. The captain positioned the aircraft for intercept of the 
runway localizer 7-10 nautical miles from the runway. A coupled approach was flown to 100 
feet altitude. The test vehicle began crossing the hold line on a taxiway near the runway 
threshold when the B757 was approximately 2000 m from the runway threshold. A few 
moments later an RTA was issued (for the RIAAS algorithm only). This is an advisory warning 
and the subject pilot was not required to take evasive action. As the aircraft approached the 
threshold, an RCA was issued for all three systems. The timing of each RCA was system- 
dependent. At this time the captain initiated a go-around maneuver following standard 
operational practices. If no alerts were received before the aircraft reached 150 ft AGL, the 
captain automatically initiated a go-around maneuver. After the RCA was issued, the test vehicle 
crossed over the runway. In some cases the pilot elected to initiate the go-around following the 
RTA. 

Scenario 2 - Departure (NASA B757)/Taxi (Test vehicle) 

Scenario 2 is illustrated in Figure 6. The captain taxied the B757 into position on the departure 
runway and held. Once the aircraft began its take off roll, the test vehicle crossed the hold line. 
The test vehicle was located at least 3000 m from the aircraft’s take off hold position. An RTA 
was issued (RIAAS algorithm only). This is an advisory warning and the subject pilot was not 
required to take evasive action. An RCA was issued as soon as an incursion had occurred or was 
eminent. The RCA occurred before the aircraft reached VI (maximum allowable rejected take 
off speed). At this time, the captain rejected the take off by stopping on the runway. After the 
RCA was issued, the test vehicle immediately crossed over the runway. In some cases due to the 
timing of the scenario, only an RCA was annunciated, skipping the intermediate RTA. 


12 





Figure 5. RIPS Scenario 1 
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Figure 6. RIPS Scenario 2 
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Scenario 3 - Taxi (NASA B757)/Det>arture (Test vehicle) 


Scenario 3, shown in Figure 7, is similar to Scenario 2 except the test vehicle is emulating a 
departing aircraft and the NASA B757 is the incurring taxi traffic. 

At the start of this test run, the B757 was positioned just behind a hold line of a taxiway that 
crosses the runway. The test vehicle accelerated to 70 mph from the departure end of the 
runway. The B757 then began crossing the hold line. An RTA was issued (for RIAAS algorithm 
only). This is an advisory warning and the subject pilot was not required to take evasive action. 
An RCA was then generated by all three systems indicating that an incursion had occurred or was 
eminent. At that time, both the B757 and test vehicle were brought to a complete stop. The test 
vehicle then exited the runway. 

Scenario 4 - Arrival (NASA B757)/Departure (Test vehicle) 

Scenario 4, illustrated in Figure 8, is similar to Scenario 1 only the test vehicle emulated a 
departing aircraft. 

The captain positioned the aircraft for intercept of the runway localizer 7-10 nautical miles from 
the runway. Coupled approaches were flown to 100 feet altitude. As the aircraft came within 
approximately 1 nm of the threshold, the test vehicle entered the runway and accelerated to 
approximately 60 kts. An RTA was issued (for RIAAS algorithm only). This is an advisory 
warning and the captain was not required to take do a go-around. An RCA was then issued by 
the three systems. At this time the captain initiated a go-around maneuver following standard 
operational practices. If no alerts were received before the aircraft reached 150 ft AGL, the 
captain automatically initiated a go-around maneuver. After the RCA, the test vehicle exited the 
runway. 

3.2 Data Collection 

The analyses contained in this report were performed using data logged by the B757 Data 
Acquisition System ( DAS). Table 3 provides a summary of the test runs. Some of the key data 
logged includes: 

• UTC time 

• Ownship position 

• Traffic identification 

• Traffic ADS-B X -Y position 

• Traffic STIS-B X - Y position 

• Traffic and Ownship Altitude 

• Traffic and Ownship Speed 

• Traffic and Ownship Heading 

• Alert status 


15 



Start, TO 


Accelerate to 70 moh 


<0 


1 

4 



Runway Traffic Alert, T1 = 10 seconds 





Figure 7. RIPS Scenario 3 
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Figure 8. RIPS Scenario 4 
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Table 3. Experiment Matrix (NASA B757) 


Test No. 

Subject 


Scenario 

Display 

Source 

Alert 

Displayed 

Gate 

False 

Alert 

Flaps 

1 

PI 

AT 

SI 

R 

Y 

N 

N 

- 

2 

PI 

AT 

S2 

I 

Y 

N 

N 

- 

3 

PI 

AT 

S3 

R 

Y 

N 

N 

- 

4 

PI 

AT 

S4 

I 

Y 

N 

N 

- 

5 

PI 

T 

SI 

G 

Y 

N 

N 

- 

6 

PI 

T 

S2 

G 

Y 

N 

N 

- 

7 

PI 

T 

S3 

I 

Y 

N 

N 

- 

8 

PI 

T 

S4 

R 

Y 

N 

N 

- 

9 

PI 

AT 

SI 

I 

Y 

N 

N 

- 

10 

PI 

AT 

S2 

R 

Y 

Y 

N 

- 

11 

PI 

AT 

S3 

G 

Y 

Y 

N 

- 

12 

PI 

AT 

S4 

G 

Y 

N 

N 

- 

13 

PI 

AT 

SI 

- 

N 

N 

N 

- 

14 

PI 

T 

SF 

- 

N 

N 

HA 

30 

15 

PI 

A 

SF 

- 

N 

N 

CA 

30 

16 

PI 

T 

SF 

- 

N 

N 

AA 

20 

17 

PI 

A 

MEL 

- 

N 

N 

HA 

30 

18 

PI 

T 

MEL 

- 

N 

N 

CA 

30 


Abbreviations: 














Subject: 

Surv Source: 

Scenario: 

Display Source: 
Alert Displayed: 
Gate: 

False Alert: 


Flaps: 


Pn (subject pilot number (n=l to 4)) 

A (Target ADS-B only), T (STIS-B only w/o ADS-B), 

AT (Target ADS-B & STIS-B w/ ADS-B) 

Sn (RI scenario number (n=l to 4)), SF (Stopping Factor 
assessment), MEL (missed exit logic) 

Algorithm driving display- R (RIAAS), I (RSM), G (GBS) 

Y (alert provided to pilot),N (alert not provided to pilot) 

Y (run starts/ends at gate), N (run does not start/end at gate) 

HA (test vehicle at hold line on ARIES arrival), CA (test vehicle cross 
runway on ARIES arrival), AA (test vehicle simulating arriving aircraft on 
ARIES arrival), N (no false alert testing) 

20 or 30 degrees 
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4.0 TEST ANALYSIS AND RESULTS 


4.1 STIS-B Surveillance Evaluation 

4.1.1 Latency 

Latencies of 2 to 6 seconds for STIS-B position reports were recorded. The average latency was 
3.5 seconds. These values exceeded the implementation goal of less than 2 seconds. A 3.5 
second latency will correspond directly to a 3.5 second delay in alerting. Latency was computed 
using ownship DGPS/inertial position and B757 STIS-B position data. The time difference 
between when ownship was reported to cross a given point and STIS-B reported the B757 as 
having crossed the same point was used as a measure of latency. Latency was sampled over 
several points. Further analysis is required to identify the source of the delays, but the key 
ground-based contributors to latency are: 

• Surveillance sensor intercommunication and processing 

• Surveillance data fusion 

• Data link processing 

• STIS-B data transmission 

4.1.2 Update Rate 

The UAT was configured to provide a STIS-B target update interval of one second. Figure 9 
provides the performance for STIS-B transmissions of ground system generated NASA B757 
track updates. The average update interval was slightly over 1 second. A separate analysis of the 
surveillance data conducted for the FAA yielded approximately the same average update interval 
of 1.06 Hz [Ref. 6]. There were periods where updates were not received for more than 10 
seconds. These gaps were not repeated in the same location from run to run, suggesting that 
STIS-B data link coverage may not have been the problem. The NASA B757 did experience 
gaps in STIS-B updates for some targets even though the data shows that the B757 continued to 
receive STIS-B data from other targets during the period of lost updates. For instance, 10 
seconds of DAL218 updates were missing even though STIS-B updates were being received on 
other traffic (file rl77stsis88). The missing updates resulted in problems for the aircraft-based 
alerting systems in several cases. 
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Update Interval (Sec) 


Figure 9. STIS-B Reception Performance 

4.1.3 Position Accuracy 

Figures 10A and 10B are plots of the STIS-B reports for the test vehicle and ownship, 
respectively, traveling along the centerline of the runway. The difference between the STIS-B 
and the ADS-B positions is approximately equal to the STIS-B cross track position error. That is 
because the ADS-B absolute error is significantly smaller than the STIS-B absolute error. The 
data in Figure 10A indicates STIS-B errors as large as 10 meters. There is an angular change in 
cross track error along the centerline. It is not obvious what might have caused this. The 
analysis of the surveillance data conducted for the FAA indicated bias errors on the fused 
surveillance data ranging up to 15 meters and standard deviations up to 6 meters [Ref. 6]. The 
magnitude of the STIS-B (fusion) errors was mostly due to the ASDE-3 radar. The reason for 
this is that the fused solution was heavily weighted by the ASDE-3 position reports. 

Figure 1 1 illustrates the along track error by showing the difference between the STIS-B reported 
position and the nose location of the B757. These measurements were made by comparing the 
ownship DGPS/inertial position (corrected to the nose) to the STIS-B B757 reported position. 
The STIS-B reported position is negative, reflecting that the reported position is behind the 
position of the nose. Some of this difference could be removed by the ground-based surveillance 
processing. Correcting the position would provide a more accurate means to detect when an 
aircraft has violated the hold line. The impact of the STIS-B position errors is relatively less 
accurate incursion alerting and greater susceptibility to false alerts. 

4.1.4 Speed Accuracy 

STIS-B speed information for the test vehicle was examined. The surveillance server had three 
surface surveillance sources for fusion to compute test vehicle position: 1090 MHz ADS-B, 1090 
MHz multilateration and ASDE-3. As described in section 4.2.5, ADS-B provides very accurate 
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X Position (meters) 


Figure 10A. Traffic STIS-B Position Accuracy Compared to ADS-B 



X Position (meters) 


Figure 10B. Ownship STIS-B Position Accuracy 
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speed information. However, the fusion processing output provided via STIS-B was of lower 
quality than the ADS-B surveillance data. At the lower speeds the data was more erratic and 
jumped significantly at times between updates on the order of 9 m/s. STIS-B traffic speeds for 
the majority of the time were very reliable at high speeds and only fluctuated by approximately 2 
m/s when traffic maintained a constant speed. Also, occasionally when the reported traffic 
positions indicated a low speed being maintained at anywhere between 5 and 10 m/s, the 
corresponding STIS-B reported speeds were much lower at around 0.5 m/s. This caused some 
problems for RIAAS in determining the correct vehicle state. RSM does not use velocity 
information provided by STIS-B. 



Figure 11. STIS-B Position Along Track Accuracy (NASA B757) 

4.1.5 Track Angle Accuracy 

Figure 12 provides a plot of the B757 track angle logged from STIS-B and the associated 
ownship’s DGPS heading as a truth source. STIS-B traffic headings incurred errors as large as 
49 degrees. In certain instances traffic headings were reported to be more than 10 degrees from 
the runway heading, which adversely affected RIAAS’s incursion algorithms. This in turn 
affected the alert processing by causing RIAAS to prematurely clear an active alert. RSM does 
not use track angle information provided by STIS-B. 



N-0OO5LOx-b>-COO5LOT-b-.COO7LO 

t-t-OJCOCO^J-^LOCOCON-I^-OO 


Seconds 


Figure 12. STIS-B Track Angle Plot (NASA B757) 
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4.1.6 Data Integrity 

STIS-B experienced a false identification problem whereby a flight ID is assigned to two 
distinctly separate ground tracks at the same time. This would generally only happen for short 
periods of time. One example is where AAL2576 was assigned to STIS-B IDs 47955 and 76 at 
the same time (file rl77stis88). These two tracks were almost 250 meters apart. Another 
example is TDX733, which was assigned to STIS-B IDs 11210 and 77 (file rl77stis88). 
Incursion alerts and associated aircraft identification is provided to the CDTI. Displaying a false 
identification during an alert situation or during normal operations is a potential safety issue. 

4.2 ADS-B Surveillance Evaluation 

4.2.1 Latency 

Truth data was not available for an accurate latency performance assessment of ADS-B. 

4.2.2 Update Rate 

ADS-B surveillance was reliable for scenarios 1 and 4, where the B757 was airborne on approach 
and the test vehicle was on the runway or taxiway. The B757’s ADS-B receiver successfully 
decoded vehicle ADS-B transmissions on average once per second. Consistent ADS-B reception 
performance was not achieved in scenarios 2 and 3 where the B757 was on the ground. During 
some runs reliable reception was achieved, while other runs had large gaps in ADS-B reception. 
The failure mechanism needs to be further investigated. 

The 1090 MHz ADS-B position reports are transmitted with an update interval of one half 
second. Figure 13 provides the ADS-B reception performance for NASA B757, receiving ADS- 
B transmissions from the ground vehicle. Based on the samples from several different runs, the 
average update interval was 1.2 seconds. There were periods where updates were not received 
for more than 10 seconds. The missing updates contributed to late alerts for the aircraft-based 
alerting systems in several cases. 
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Figure 13. ADS-B Reception Performance of Test Vehicle Transmissions 


4.2.3 Position Accuracy 

ADS-B reports are more accurate than STIS-B. In comparison to the STIS-B plot (Figure 10), 
ADS-B provides test vehicle position reports that more precisely follow centerline (Figure 16) 
while the vehicle is traveling along the centerline. Along track errors will be similar to the cross 
track errors shown. ADS-B position report accuracy is nearly equivalent to the accuracy of 
LAAS. An analysis of the DFW LAAS indicated 95% position accuracy of approximately 2 
meters [Ref. 7], The benefit of better ADS-B accuracy as compared to STIS-B should be more 
accurate runway incursion alerting and minimization of false alerts. 

4.2.4 Speed Accuracy 

ADS-B speed data was more accurate than STIS-B as illustrated in Figure 15. The ADS-B 
reported speeds were more consistent with normal operation of a vehicle than the STIS-B 
reported speeds. The STIS-B appeared to have a problem accurately reporting speeds below 15 
m/s. There were still times when errors approaching 4 meters/sec were evident at the lower 
speeds of less than 8 meters/sec. 

4.2.5 Track Angle Accuracy 

ADS-B track angle data was fairly consistent the majority of the time, and had little adverse 
effect during alert processing. As noted with the STIS-B data, occasional track angle jumps in 
the data had the potential to affect the RIAAS incursion alerting. Up until the time of the 
demonstration, there was a software error on the B757 systems, which caused some headings to 
be reported with a 180-degree error. This error caused RIAAS to erroneously determine 
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entry/exit of the test vehicle from the runway safety zone and erroneously declare the heading of 
the vehicle on the runway. Occasionally, this error resulted in delayed alerts and toggling on/off 
of alerts for RIAAS, which uses ADS-B heading information. Figure 14 provides a comparison 
of ADS-B versus STIS-B track angle data for the test vehicle after the ADS-B problem was 
fixed. Similar to the comparison of ownship DGPS to STIS-B, figure 12, the ADS-B track angle 
data was smoother than the STIS-B data. The figure also shows that the ADS-B Rack angle plot 
leads the STIS-B track angle information reflecting either a lower latency of ADS-B over STIS-B 
or STIS-B track processing introduced delays. 



COt— CQx— COx— COt— COt— 
x-x-CNCNOOOO^J-^tLO 


Samples (one second intervals) 


Figure 14. ADS-B vs. STIS-B Track Angle for Test Vehicle 



Figure 15. ADS-B versus STIS-B Speed for Test vehicle 
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4.2.6 Data Integrity 

ADS-B provided a high level of integrity with respect to traffic information and identification. 
The traffic ID problems identified with STIS-B were not experienced with ADS-B. The only 
integrity related problem experienced with the ADS-B data was due to incorrect processing of the 
heading information on-board the NASA B757. 

4.3 Ownship Navigation Evaluation 

4.3.1 LAAS Accuracy 

A LAAS system was used to provide the NASA B757 with highly accurate position information. 
Preliminary test results at DFW indicated position accuracies on the order of 1-2 meters [5], 
This accuracy is achieved by blending LAAS position information with an on-board inertial 
guidance system. RIAAS applies a position offset of 22 meters to the ownship reported position 
to determine the location of the nose of the aircraft. This correction provides for more timely 
alerts in scenarios where the B757 violates the hold lines. It is estimated that the correction 
provides for reliable alerting almost 4 seconds sooner than if no correction were performed. The 
B757 also has highly accurate speed and track information. 

4.3.2 Conclusions 

Although complete performance data analysis was not available at the time this report was 
written, the initial conclusion is that the position, speed, and track performance was more than 
adequate to support the implementation of RIAAS and RSM. Alert response time performance 
benefited from the position offset correction for ownship. This capability was not provided for 
ADS-B and STIS-B traffic position updates. The position accuracy should support reliable 
determination of the aircraft tail location. Knowledge of tail location can be used to determine 
when aircraft are clear of the runway safety zone. 

4.4 Runway Incursion Alerting Evaluation 

4.4.1 RIAAS Alerting Performance 

RIAAS performed as designed throughout testing. Alerts were accurate and timely. Out of 47 
runs, RIAAS properly alerted on 44 of them. The three missed alerts were caused by missing and 
erroneous traffic data. A small number of RIAAS alerts were issued late. This was directly 
attributable to gaps in both STIS-B and ADS-B data. RIAAS generated two false alerts over the 
course of testing, both directly caused by erroneous traffic data. 

4.4.1. 1 RIAAS Scenario Alert Plots 

Figures A-l through A-4 are plots of actual RIAAS runs, performed at DFW during testing. 
These plots represent typical RIAAS performance with good traffic data and proper scenario 
timing. Scenarios 1-3 were ADS-B/STIS-B runs, while scenario 4 was STIS-B-only. 
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Figure A-l is a plot of a Scenario 1 run, representing an Arrival/Taxi situation. As ownship gets 
within 2000 meters of the runway threshold and traffic pulls across the hold line, an RTA is 
issued, providing a heightened measure of caution. Ownship continues as if landing. As 
ownship gets within 1000 meters of the threshold, an RCA is issued. A few seconds later, 
ownship initiates a go-around at 172 ft altitude, safely clearing traffic and a potential collision. 
The vehicle proceeds across the runway, ending the scenario. 

Figure A-2 is a plot of a Scenario 2 run, representing a Departure/Taxi situation. Ownship pulls 
onto the runway and proceeds to begin its take off roll. Once ownship has reached a minimum 
departure speed, traffic rolls across the hold line, and an RTA is issued, providing a heightened 
awareness of a potential conflict. Traffic continues crossing toward the runway. Finally, an 
RCA is issued as traffic crosses more than 15 meters over the hold line. Ownship is Raveling at 
60 kts at this point. A few moments later, ownship initiates an aborted take off, slowing to taxi 
speed and safely pulling off the runway with over 2000 meters separation remaining. 

Figure A-3 is a plot of a Scenario 3 run. This is very similar to Scenario 2, only ownship and 
traffic switch roles. Traffic pulls onto the runway and proceeds to begin its take off roll. Once 
traffic has reached a minimum “high-speed” state, ownship rolls across the hold line and an RTA 
is issued, providing a heightened awareness of a potential conflict. Ownship continues crossing 
toward the runway. Finally, an RCA is issued as ownship crosses more than 10 meters over the 
hold line. Traffic is traveling at 57 kts at this point, with a separation distance of 31 10 meters. 
Ownship comes to a halt in response to the RCA, stopping a safe distance from the runway and 
avoiding a potential collision with the departing traffic. 

Figure A-4 is a plot of a Scenario 4 run, representing an Arrival/Departure situation. Ownship is 
preparing for an arrival. Traffic crosses the hold line and an RTA is issued, providing a 
heightened awareness of a potential conflict. Traffic continues onto the runway to initiate its take 
off roll. As ownship gets within 600 meters of the runway threshold, an RCA is issued. 
Ownship initiates a go-around at 217 meters altitude, and safely avoids a potential collision with 
the departing traffic. 

4.4.1. 2 RIAAS Alert Performance Summary 

RIAAS alerts are summarized in Tables 4 through 7, and alert data is presented in Tables 8 
through 19 by scenario and alert type (RTAs and RCAs). Time between first RTA and first RCA 
is also documented when applicable. Due to timing of the scenarios, single alerts (RTA-only or 
RCA-only) were received in some runs. 

RIAAS performed as designed throughout all of DFW testing. Problems with alerting were all 
attributed to erroneous data and data drop-outs associated with STIS-B and ADS-B operation. 
RIAAS exhibited a 100% success rate for runs in which reliable and accurate traffic data was 
supplied. Alert timing results varied due to timing of the scenarios (i.e. vehicle movement 
relative to ownship movement) as can be expected. A comprehensive summary of RIAAS 
alerting performance is found in Appendix B. 
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Scenario 1 


Tables 4, and 8 through 10 represent alert data for all RIAAS Scenario 1 runs. Important 
variables for this scenario are ownship distance to threshold, traffic distance to hold line and 
traffic distance to runway. 

As shown in Table 8, the average ownship distance to threshold at the time of the first RTA alert 
was 1839.2 meters before the runway threshold. Negative distance values indicate that the 
aircraft had not crossed the threshold when the alert occurred. The RTAs occurred within 1967.7 
and 1498.9 meters from threshold. The average traffic distance over the hold line at the time of 
the first RTA was 19.6 meters. Even at the maximum distance over the hold line, traffic was not 
on the runway. One run from the 16 th , Matrix Run #1 was labeled an outlier. The alert was 
received late relative to the rest of the RTAs, resulting from bad scenario timing. Bad scenario 
timing generally occurs when the vehicle is either too early or too late in crossing the hold 
line/runway edge, or when vehicle speeds are too fast or too slow at specific points throughout 
the run. 

As shown in Table 9, the average ownship distance to threshold at the time of the first RCA alert 
was 964.6 meters before the runway threshold. At the minimum, ownship distance to threshold 
was 941 meters when a RCA occurred. In all cases, traffic was on the runway, indicated by a NA 
in the traffic distance to hold line column, when the RCA occurred. However, sufficient 
advanced alerting was provided for ownship to safely perform a go-around. Three runs were 
labeled outliers in the RCA study. The first, from October 16 th , Matrix Run #1 is the same run 
that was labeled an outlier for the RTA study, and was a result of bad scenario timing. The 
second outlier, from October 16 th , Matrix Run #9, was the result of a false alert, caused by bad 
ADS-B data. The alert came on prematurely in response to this erroneous data, then cleared 
before legitimately alerting. The final outlier is from October 17 th , Matrix Run #19. The timing 
of this scenario was off a bit, as traffic had actually crossed the runway completely before 
ownship was close enough to the runway threshold to cause an RCA. 

As shown in Table 10, time between first RTA and first RCA was calculated and compared for 
each run in which both an RTA and an RCA were received. For Scenario 1, all runs resulted in 
both an RTA and an RCA. The average time between alerts was 13.5 seconds. There was one 
outlier for this study; Matrix Run #1 from October 16 th . This run had improper alerts due to the 
scenario timing. The average value of 13.5 seconds appears to be consistent with the predicted 
results for this scenario. Figure A-l shows typical RIAAS alerting performance for Scenario 1. 

Scenario 2 


Tables 5 and 11 through 13 represent alert data for all RIAAS Scenario 2 runs. Important 
variables for this scenario are ownship speed, separation distance, traffic distance to hold line and 
traffic distance to runway. 

As shown in Table 11, the average ownship departure speed at the time of the first RTA was 30.2 
m/s or 59 kts. The maximum speed at time of alert was 37.4 m/s or 73 kts. The average 
separation distance at the time of the first RTA was 3223.5 meters. The average traffic distance 
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over the hold line at the time of the RTA was 10.8 meters. There was one outlier for Scenario 2 
RTA’s. Matrix Run #6 from October 17 th , resulted in a late alert. It appears as though this is a 
scenario timing issue, caused by a late start for the vehicle. 

As shown in Table 12, the average ownship speed at the time of the first RCA was 33.8 m/s or 66 
kts with a maximum speed of 44.5 m/s or 87 kts. The average separation distance at the time of 
the first RCA alert was 3167.7 meters with a minimum value of 3054 meters. The RCA occurred 
well before traffic entered the runway. The average traffic distance to hold line at the time of the 
first RCA was 26.2 meters, with a maximum value of 39.9 meters. There was one outlier for this 
set. Matrix Run #56 on October 20 th resulted in a late alert. Traffic ADS-B data did not update 
for a period of time, while the vehicle was moving. Once the data updated, RIAAS alerted 
properly. 

Five Scenario 2 runs resulted in multiple alert types. In four of these runs, the RTA was received 
before the RCA. In the remaining run, improper timing of the scenario caused the RCA to come 
first, followed by the RTA. This run was considered an outlier, and was not used to calculate 
time between alert values. 

Table 13 shows the average time between the first RTA and the first RCA was 1.7 seconds, with 
a maximum value of 3.1 seconds and a minimum of 0.9 seconds. Figure A-2 shows typical 
RIAAS alerting performance for Scenario 2. 

Scenario 3 


Tables 6 and 14 through 16 represent alert data for all RIAAS Scenario 3 runs. Important 
variables for this scenario are ownship distance to hold line, ownship distance to runway edge, 
traffic speed and separation distance. 

Table 14 shows that the average ownship distance to hold line at the time of the first RTA alert 
was 1.2 meters over the hold line. The maximum distance over the hold line is 1.5 meters. The 
average traffic speed at the time of the first RTA was 28.6 m/s or 56 kts. The average separation 
distance at the time of the first RTA was 2710.3 meters. There were no outliers in the RTA study 
for Scenario 3. 

As shown in Table 15, the average ownship distance to hold line for Scenario 3 RCA’s at the 
time of the first alert was 25 meters over the hold line. The maximum value for ownship 
distance to hold line was 36.0 meters. The average traffic speed at the time of the first alert was 
28.7 m/s or 56 kts, with a maximum value of 34 m/s or 66 kts. The average separation distance 
at the time of the first alert was 2844.3 meters, with a maximum value of 3195 meters. Timing 
of scenarios for Matrix #s 21 and 25 from October 17 th , and Matrix #39 from October 18 th all 
resulted in RCA’s only. Traffic was early in executing each run. As a result, when ownship got 
within the alerting range, the ensuing alert was an RCA instead of an RTA as traffic was already 
on the runway. These three runs were eliminated from this study as they were considered 
outliers. Looking at separation distance for all of the runs in Table 6, it is obvious that scenario 
timing was not consistent for Scenario 3 throughout testing as separation distance at the time of 
the first alert varied widely with each run. 
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Three Scenario 3 runs resulted in multiple alert types, while seven runs resulted in RCA’s only. 
In all of the multiple-alert runs, the RTA was received before the RCA. The average time 
between alerts was 6 seconds with a maximum value of 1 1 . 1 seconds and a minimum value of 
2.5 seconds. The results varied a considerable amount, again, due to varied timing of the 
scenarios. Based on these variations, the resulting time between alerts looks acceptable. Results 
can be seen in Table 16. Figure A-3 shows typical RIAAS alerting performance for Scenario 3. 

Scenario 4 


Tables 7 and 17 through 19 represent alert data for all RIAAS Scenario 4 runs. Important 
variables for this scenario are ownship distance to threshold, separation distance, distance to 
runway (edge) and traffic speed. 

Table 17 shows that the average ownship distance to threshold at the time of the first RTA was 
1853.1 meters. The maximum value at the time of the first alert was 1982.2 meters and the 
minimum value was 1544.4 meters. The average separation distance at the time of the first RTA 
was 3610.3 meters. The average traffic distance to runway was 70.1 meters from the runway 
edge while still off the runway. Two runs were eliminated from this study as outliers. Matrix 
Run #26 from October 18 th was eliminated due to bad scenario timing, as the first alert was 
received while the vehicle was just barely off the runway, and traveling at a speed three times 
greater than average for this scenario. Matrix Run #30, also from October 18 th , was eliminated 
again due to bad scenario timing. The vehicle was on the runway at the time of the alert, 
traveling more than four times the average for this scenario. All other values were as expected 
based on RIAAS design criteria and scenario timing. 

RIAAS RCA data is found in Table 18. The average ownship distance to threshold at the time of 
first RCA was 478.7 meters, with a minimum distance of 254.9 meters. The average separation 
distance at the time of the first alert was 2635.6 meters, with a minimum distance of 2343 
meters. The average traffic distance to runway edge at the time of the first alert was 18 meters 
over (on the runway). Matrix Run #8 from October 16 th was eliminated from this study as an 
outlier. The timing of the scenario resulted in an early alert, while the vehicle was traveling at 
about half the average speed for this scenario. All other values were as expected. 

Seven Scenario 4 runs resulted in multiple alert types. The average time between the first RTA 
and the first RCA was 21 seconds. The standard deviation was 3.1 seconds, with a maximum 
value of 26 seconds and a minimum value of 16.9 seconds. Three Scenario 4 runs resulted in 
RTA’s only. This was caused by early initiation of go-arounds, as pilots either responded to one 
of the other systems which happened to alert before a RIAAS RCA was generated, or they 
initiated a go-around in response to a RIAAS RTA as opposed to waiting for the RCA. Values 
were as expected here, and can be seen in Table 19. Figure A-4 shows typical RIAAS alerting 
performance for scenario 4. 

RIAAS False Alerts 

Not all false alerts were received during RIPS incursion runs. Every false alert captured in the 
DAS data has been noted, regardless of the type of test being performed. RIAAS generated only 


31 



two false alerts throughout all of DFW testing. Both false alerts were direct results of erroneous 
data. Table H-l lists the scenario information and traffic/ ownship positions at the time of each 
false alert. In one case, traffic was traveling away from ownship at a high rate of speed. Traffic 
heading suddenly flipped 180 degrees indicating that it was headed straight toward ownship, 
resulting in a RIAAS alert. The second case of a false alert occurred toward the end of a 
successful run. Traffic had cleared the runway and crossed the hold line going away from the 
runway. On the very next data update, erroneous position data indicated that traffic was 50 
meters closer to the runway than it in fact was, causing RIAAS to falsely re-issue the alert it had 
just cleared. 


Table 4. Scenario 1 RIAAS Alert Summary 


Date 

Matrix 

# 

■ 


Alert 

On- 

Time 

Alert 

Late 

False 

Alert 

YES 

False 

Alert 

NO 

Notes 

16-Oct 

1 

RTA 


XX 



XX 

Alerts OK. 

16-Oct 

1 

RCA 


XX 



XX 


16-Oct 

9 

RTA 


XX 


XX 


Alerts OK. Bad ADS-B data. 

16-Oct 

9 

RCA 




XX 



17-Oct 

5 

RTA 



XX 


XX 

Late Alerts (approximately 3-4 sec). 

17-Oct 

5 

RCA 


XX 



XX 


17-Oct 

19 

RTA 


XX 


XX 


Alerts OK. False Alert - due to bad position 
data. 

17-Oct 

19 

RCA 


XX 


XX 



18-Oct 

23 

RTA 





XX 

Alerts OK. 


23 

RCA 





XX 



27 

None 






Traffic data indicates no van movement. 


45 

RTA 





XX 

Alerts OK. 


45 

RCA 





XX 



37 

RTA 





XX 

Alerts OK. 


37 

RCA 





XX 



41 

RTA 





XX 

Alerts OK. 


41 

RCA 





XX 



63 

RTA 


XX 



XX 

Alerts OK. 


63 

RCA 


XX 



XX 



55 

RTA 


XX 



XX 

Alerts OK. 


55 

RCA 


XX 



XX 



59 

RTA 


XX 



XX 

Alerts OK. 


59 

RCA 


XX 



XX 
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Table 5. Scenario 2 RIAAS Alert Summary 


Date 

Matrix 

# 

■ 


Alert 

On- 

Time 

Alert 

Late 

False 

Alert 

YES 

False 

Alert 

NO 

Notes 

16-Oct 

10 

RCA 


XX 



XX 

Alert OK. 

17-Oct 

2 

RTA 


XX 



XX 

Alerts OK. Bad position data. 

17-Oct 

2 

RCA 


XX 



XX 


17-Oct 

6 

RCA 



XX 


XX 

Alerts OK 

17-Oct 

6 

RTA 


XX 



XX 


17-Oct 

20 

RTA 


XX 



XX 

Alerts OK. 

17-Oct 

20 

RCA 


XX 



XX 


17-Oct 

24 

RCA 


XX 



XX 

Alerts OK. 

18-Oct 

28 

RCA 





XX 

Alert OK. 

18-Oct 

42 

RTA 





XX 

Alerts OK. 

18-Oct 

42 

RCA 





XX 


18-Oct 

38 

RTA 





1 XX i 

Alerts OK. 

18-Oct 

38 

RCA 





XX 


19-Oct 

46 

RCA 





XX 

Alert OK. 

20-Oct 

56 

RCA 



XX 


XX 

Late Alert. Bad ADS-B data. 

20-Oct 

64 

RTA 


XX 



XX 

Alerts OK. 

20-Oct 

64 

RCA 


XX 



XX 



Table 6. Scenario 3 RIAAS Alert Summary 


Date 

Matrix 

# 

a 


Alert 

On- 

Time 

Alert 

Late 

False 

Alert 

YES 

False 

Alert 

NO 

Notes 

16-Oct 

3 

RTA 


XX 



XX 

Alerts OK. 

16-Oct 

3 

RCA 


XX 



XX 


16-Oct 

11 

RTA 


XX 


XX 


Alerts OK. Data gaps. Bad data. 

16-Oct 

11 

RCA 


XX 


XX 



17-Oct 

7 

RCA 


XX 



XX 

Alert OK. 

17-Oct 

21 

RCA 


XX 



XX 

Alert OK. 

17-Oct 

25 

RCA 


XX 



XX 

Alert OK. 

17-Oct 

29 

RCA 



XX 


XX 

Alert OK. Data gaps 

18-Oct 

47 

RCA 





XX 

Alert OK. Data gap 

18-Oct 

39 

RCA 





XX 

Alerts OK. 

19-Oct 

43 

None 






No STIS-B data received. 

20-Oct 

57 

RTA 


XX 



XX 

Alerts OK. 

20-Oct 

57 

RCA 


XX 



XX 


20-Oct 

65 

RCA 


XX 



XX 

Alert OK. Data gaps 

20-Oct 

61 

None 






No STIS-B data received 
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Table 7. Scenario 4 RIAAS Alert Summary 


False False 
Alert Alert 



16-Oct 


16-Oct 


17-Oct 


17-Oct 


17-Oct 


18-Oct 


18-Oct 


18-Oct 


18-Oct 


18-Oct 


18-Oct 


19-Oct 


19-Oct 


20-Oct 


20-Oct 


20-Oct 


20-Oct 


20-Oct 


20-Oct 



lerts OK. 


lerts OK. Data gaps 



lertOK. 


lerts OK. Bad heading data 



lertOK. 


lertOK. 


lerts OK. 


lerts OK. 


lerts OK. 


XX Late Alerts (approximately 2 sec) 



Table 8. Scenario 1 RIAAS RTAs 


Scenario 1 RTA 


UTC 


24954.5 


20628.3 


32227.2 


19077.7 


28680.3 


31388.3 


18534.4 


20959.8 


29977.5 


32729.8 


Matrix Run # 


9 


5 


19 


23 


45 


37 


41 


63 


55 


59 


AVERAGE 


STDEV 


MINIMUM 


MAXIMUM 



-1824.7 


-1498.9 


-1904.9 


-1732.9 


-1964.3 


-1884 


-1745.1 


-1953.1 


-1916.1 


-1967.7 


-1839.2 


146.6 


-1498.9 


-1967.7 
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Table 9. Scenario 1 RIAAS RCAs 


Scenario 1 RCA 

Date 

UTC 

Matrix Run # 

HuHCHi 

Traffic Dist. To 
H.L. (m) 

Traffic Dist. 
To RWY (m) 

17-Oct 

20636.1 

5 

-975.9 

NA 

19.4 

18-Oct 

19089.8 

23 

-965.4 

NA 

9.3 

18-Oct 

28695.0 

45 

-969.3 

NA 

3.8 

18-Oct 

31402.0 

37 

-968.7 

NA 

3.9 

19-Oct 

18546.2 

41 

-976.6 

NA 

9.6 

20-Oct 

20974.6 

63 

-951.3 

NA 

3.8 

20-Oct 

29993.3 

55 

-968.2 

NA 

20.3 

20-Oct 

32746.5 

59 

-941 

NA 

5.7 


AVERAGE 

-964.6 

NA 

9.5 

STDEV 

12.3 

NA 

6.8 

MINIMUM 

-941 

NA 

3.8 

MAXIMUM 

-976.6 

NA 

20.3 


Note - Distances to threshold, runway, and hold line have positive values for locations across these points and 
negative values for distances further away. 


Table 10. Scenario 1 Time Between RIAAS Alerts 


Scenario 1 Time between first RTA and first RCA 

Date 

Matrix Run # 

Time Between First 
RTA and First RCA 
(s) 

16-Oct 

9 

13.0 

17-Oct 

5 

7.9 

17-Oct 

19 

15.0 

18-Oct 

23 

11.6 

18-Oct 

45 

14.9 

18-Oct 

37 

14.1 

19-Oct 

41 

12.1 

20-Oct 

63 

15.1 

20-Oct 

55 

14.9 

20-Oct 

59 

16.6 


AVERAGE 

13.5 

STDEV 

2.5 

MINIMUM 

7.9 

MAXIMUM 

16.6 
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Table 11. Scenario 2 RIAAS RTAs 


Scenario 2 RTA 

Date 

UTC 

Matrix Run # 

gum 


mnssH 

Traffic Dist. To 
RWY (m) 

17-Oct 

30971.07 

20 

32.6 

3227.0 

14.1 

-55.4 

18-Oct 

27461.46 

42 

37.4 

3133.0 

13.4 

-64.1 

18-Oct 

32511.49 

38 

25.9 

3254.0 

9.9 

-67.6 

20-Oct 

33871.66 

64 

24.7 

3280.0 

5.6 

-71.9 


AVERAGE 

30.2 

3223.5 

10.8 

-64.8 

STDEV 

6.0 

64.1 

3.9 

7.0 

MINIMUM 

24.7 

3133.0 

5.6 

-55.4 

MAXIMUM 

37.4 

3280.0 

14.1 

-71.9 


Table 12. Scenario 2 RIAAS RCAs 


Scenario 2 RCA 

Date 

UTC 

Matrix Run # 

Ownship 

Speed 

(m/s) 

Separation Dist. (m) 


fmSSmm 

16-Oct 

33848.17 

10 

15.9 

3242.0 

37.5 

-32.0 


22952.3 

2 

30.6 

3220.0 

16.2 

-53.2 


24305.21 

6 

42.0 

3054.0 

30.5 

-39.0 

17-Oct 

30973.04 

20 

38.9 

3158.0 

31.2 

-38.3 

17-Oct 

35895.96 

24 

44.5 

3098.0 

24.0 

-45.4 

18-Oct 

22760.77 

28 

33.1 

3211.0 

21.3 

-56.2 

18-Oct 

27462.44 

42 

40.6 

3097.0 

19.9 

-57.7 

18-Oct 

32514.44 

38 

34.2 

3163.0 

39.9 

-37.7 

19-Oct 

24933.63 

46 

29.2 

3183.0 

22.0 

-55.5 

20-Oct 

33872.65 

64 

28.6 

3251.0 

19.9 

-57.7 


AVERAGE 

33.8 

3167.7 

26.2 

-47.3 

STDEV 

8.4 

66.8 

8.1 

9.9 

MINIMUM 

15.9 

3054.0 

16.2 

-32.0 

MAXIMUM 

44.5 

3251.0 

39.9 

-57.7 
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Table 13. Scenario 2 Time Between RIAAS Alerts 


Scenario 2 Time between first RTA and first RCA 

Date 

Matrix Run # 

Time Between 
RTA and RCA (s) 

17-Oct 

20 

1.9 

18-Oct 

42 

0.9 

18-Oct 

38 

3.1 

20-Oct 

64 

1.0 


AVERAGE 

1.7 

STDEV 

1.0 

MINIMUM 

0.9 

MAXIMUM 

3.1 


Table 14. Scenario 3 RIAAS RTAs 


Scenario 3 RTA 

Date 

UTC 

Matrix Run # 



IIHH 

Separation 
Dist. (m) 


23013.77 

3 

1.5 

-76.0 

31.7 

2221.0 


33480.24 

11 

1.2 

-76.3 

26.8 

2713.0 

20-Oct 

17479.9 

57 

0.9 

-67.5 

27.3 

3197.0 


AVERAGE 

1.2 

-73.3 

28.6 

2710.3 

STDEV 

0.3 

5.0 

2.7 

488.0 

MINIMUM 

0.9 

-67.5 

26.8 

2221.0 

MAXIMUM 

1.5 

-76.3 

31.7 

3197.0 


37 













































Table 15. Scenario 3 RIAAS RCAs 


Scenario 3 RCA 

Date 

UTC 

Matrix Run # 

Ownship Dist. 
To H.L. (m) 

Ownship Dist. 
To RWY (m) 

T raffic 
Speed 
(m/s) 

Separation 
Dist. (m) 


23018.69 

3 

11.8 

-65.7 




33491.06 

11 

33.5 

-44.0 

27.8 


17-Oct 

19569.06 

7 

26.8 

-47.8 

22.6 

3188.0 

17-Oct 

34254.3 

29 

30.6 

-44.0 

26.8 

3195.0 

18-Oct 

30252.03 

47 

25.0 

-44.5 

29.3 

2933.0 

20-Oct 

17481.87 

57 

10.5 

-57.9 

34.0 

3110.0 

20-Oct 

19934.05 

65 

36.9 

-32.5 

31.6 

2979.0 


AVERAGE 

25.0 

-48.1 

28.7 

2844.3 

STDEV 

10.3 

10.8 

3.6 

424.9 

MINIMUM 

10.5 

-32.5 


2102.0 

MAXIMUM 

36.9 

-65.7 




Table 16. Scenario 3 Time Between RIAAS Alerts 


Scenario 3 Time between first RTA and first RCA 

Date 

Matrix Run # 

Time Between 
RTA and RCA (s) 

16-Oct 

3 

4.3 

16-Oct 

11 

11.1 

20-Oct 

57 

2.5 


AVERAGE 

6.0 

STDEV 

4.5 

MINIMUM 

2.5 

MAXIMUM 

11.1 
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Table 17. Scenario 4 RIAAS RTAs 


Scenario 4 RTA 

Date 

UTC 

Matrix Run # 

Ownship 
Dist. To 
T.H. (m) 

Separation 
Dist. (m) 

T raffic 
Dist. To 
RWY 
(m) 

Traffic 

Speed 

(m/s) 

16-Oct 

21419.44 

4 

-1855.9 

3526.0 

-79.3 

5.1 

16-Oct 

24191.03 

8 

-1544.4 

3210.0 

-70.8 

6.7 

17-Oct 

21474.46 

12 

-1878.5 

3550.0 

-79.3 

5.7 

17-Oct 

29449.51 

22 

-1982.2 

3652.0 

-72.2 

6.7 

18-Oct 

29301.27 

44 

-1825.8 

3640.0 

-67.2 

7.7 

18-Oct 

30897.48 

40 

-1926.2 

3783.0 

-76.4 

5.7 

19-Oct 

21485.25 

48 

-1932.4 

3750.0 

-72.2 

6.7 

20-Oct 

18464.35 

58 

-1953.8 

3755.0 

-74.3 

6.2 

20-Oct 

21833.35 

62 

-1696.8 

3504.0 

-42.9 

8.7 

20-Oct 

30635.6 

66 

-1934.9 

3733.0 

-66.4 

7.2 


AVERAGE 

-1853.1 

3610.3 

-70.1 

6.6 

STDEV 

135.9 

173.3 

10.5 

1.1 

MINIMUM 

-1544.4 

3210.0 

-42.9 

5.1 

MAXIMUM 

-1982.2 

3783.0 

-79.3 

8.7 


Table 18. Scenario 4 RIAAS RCAs 


Scenario 4 RCA 

Date 

UTC 

Matrix Run # 

Ownship Dist. 
To T.H. (m) 

Separation 
Dist. (m) 


T raffic 
Speed 
(m/s) 

16-Oct 

21442.07 

4 

-254.9 

2343.0 

21.7 

31.4 

17-Oct 

21494.15 

12 

-578.4 

2561.0 

19.8 

26.2 

18-Oct 

18505.7 

26 

-500.8 

2720.0 

6.3 

34.3 

18-Oct 

21677.83 

30 

-576.7 

2733.0 

14.4 

28.8 

19-Oct 

21506.9 

48 

-558.5 

2751.0 

21.1 

29.8 

20-Oct 

18486.14 

58 

-513.3 

2656.0 

21.8 

27.5 

20-Oct 

21850.08 

62 

-574.5 

2707.0 

17.8 

26.8 

20-Oct 

30662.17 

66 

-272.6 

2614.0 

21.3 

30.4 


AVERAGE 

-478.7 

2635.6 

18.0 

29.4 

STDEV 

135.9 

134.7 

5.4 

2.7 

MINIMUM 

-254.9 

2343.0 

6.3 

26.2 

MAXIMUM 

-578.4 

2751.0 

21.8 

34.3 
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Table 19. Scenario 4 Time Between RIAAS Alerts 


Scenario 4 Time between first RTA and first RCA 

Date 

Matrix Run # 

Time Between 
Alerts (s) 

16-Oct 

4 

23.0 

17-Oct 

12 

20.0 

18-Oct 

26 

17.7 

19-Oct 

48 

22.0 

20-Oct 

58 

21.2 

20-Oct 

62 

16.9 

20-Oct 

66 

26.0 


AVERAGE 

21.0 

STDEV 

3.1 

MINIMUM 

16.9 

MAXIMUM 

26.0 


40 































4.4.2 RSM Alerting Performance 

RSM exhibited reliable and consistent alerting performance when accurate and reliable traffic 
information was available. It appears as though all problems with alerting were attributed to 
erroneous data, data drop-outs, or other data-related problems. Results varied from run to run 
depending on the timing of each individual scenario, as can be expected. 

4.4.2. 1 RSM Scenario Alert Plots 

Figures C-l through C-4 are plots of actual RSM runs, performed during testing at DFW. These 
plots represent typical RSM performance with good traffic data and proper scenario timing. 
Scenarios 1-4 were ADS-B/STIS-B runs. 

Figure C-l is a plot of a Scenario 1 ran, representing an Arrival/Taxi situation. As ownship gets 
within 1 nm of the runway threshold and traffic is 16 m across the hold line, an RCA is issued. 
A few seconds later, ownship initiates a go-around at 326 ft altitude, safely clearing traffic and a 
potential collision. Traffic proceeds across the runway, ending the scenario. 

Figure C-2 is a plot of a Scenario 2 run. This scenario represents a Departure/Taxi situation. 
Ownship pulls onto the runway and proceeds to begin its take off roll. Once ownship has 
reached a minimum departure speed, traffic rolls across the hold line and continues toward the 
runway. As traffic is almost 40 m over the hold line, an RCA is generated with vehicle 
separation distance of 3086 m. Ownship is traveling at 78 kts at this point. Three seconds later, 
ownship initiates an aborted take off, slowing to taxi speed and safely pulling off the runway. 

Figure C-3 is a plot of a Scenario 3 ran. This is very similar to Scenario 2, with ownship and 
traffic switching roles. Traffic pulls onto the runway and proceeds to begin its take off roll. 
Once traffic has reached a minimum take off speed, ownship begins to roll across the hold line. 
Ownship continues crossing toward the runway. An RCA is issued as ownship passes 30 meters 
over the hold line and vehicle separation is 2265 meters. Traffic is traveling at 58 kts at the time 
of the alert. Ownship comes to a halt in response to the RCA, stopping a safe distance from the 
runway edge and avoiding a potential collision with the departing traffic. 

Figure C-4 is a plot of a Scenario 4 run. This scenario represents an Arrival/Departure situation. 
Ownship is preparing for an arrival. Traffic crosses the hold line and continues toward the 
runway. As ownship nears 1700 meters from the runway threshold, an RCA is issued. Traffic is 
still off the runway, 36 meters from runway edge. Traffic continues onto the runway to initiate 
its take off roll. Ownship initiates a go-around at 282 ft altitude, and safely avoids a potential 
collision with the departing traffic. 
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4.4.2.2 RSM Alert Performance Summary 

RSM alerts are summarized in Tables 20 through 23, and alert data is presented in Tables 24 
through 27 and is sorted by scenario. A comprehensive summary of RSM alerting performance 
is found in Appendix D. 

Scenario 1 


Table 24 presents RCA alert results for RSM Scenario 1 runs. Important variables for this 
scenario are ownship distance to threshold, traffic distance to hold line, and traffic distance to 
runway edge. As shown in Table 24, the average ownship distance to threshold at the time of the 
first RCA was 1753.6 meters before the runway threshold. The alerts occurred while ownship 
distance to threshold values ranged from 1426.2 to 1922.9 meters. The average traffic distance 
over the hold line at the time of the first alert was 30.8 meters. For valid Scenario 1 runs, all of 
the alerts began before traffic reached the runway. Three runs were labeled outliers in the 
scenario 1 RSM alert analysis. The first, from October 16 th , Matrix Run #1, was a result of bad 
scenario timing. The second outlier, Matrix Run #41, flown on October 19 th appeared to be a 
result of timing as well, as the alert was not generated until traffic was on the runway. The final 
outlier was Matrix Run #27 from October 1 8 lh . An RSM alert was not generated during this run 
due to erroneous ADS-B data. The target did appear but its indicated position did not change 
throughout the run, while the vehicle was in fact moving. 

Scenario 2 


Table 25 presents RCA alert results for RSM Scenario 2 runs. Important variables for this 
scenario are ownship speed, separation distance, traffic distance to hold line and traffic distance 
to runway edge. As shown in Table 25, the average ownship departure speed at the time of the 
first RCA was 37.8 m/s or 73 kts, with a maximum recorded speed of 50.3 m/s or 98 kts. The 
average separation distance at the time of the first alert was 3135.5 meters. The average traffic 
distance over the hold line at the time of alert was 29.6 meters. Matrix Run #56 from October 
20 th was omitted as an outlier. RSM did not alert on this run due to missing traffic data. ADS-B 
did not update for 30 seconds during this run. 

Scenario 3 


Table 26 presents alert data for RSM Scenario 3 runs. Important variables for this scenario are 
ownship distance to hold line, ownship distance to runway edge, traffic speed and Separation 
distance. As shown in Table 26, the average ownship distance to hold line at the time of the first 
alert was 28 meters. Ownship never reached the runway at the time of the first RSM alert for any 
valid scenario 3 runs. The average traffic departure speed at the time of the first alert was 23.6 
m/s or 46 kts. The maximum speed recorded at the time of the first alert was 30.4 m/s or 59 kts. 
The average separation distance at the time of first alert was 2973.6 meters. Traffic and ownship 
were never closer than 2072.2 meters at the time of the first RSM alert. 

Two runs were eliminated from the Scenario 3 RSM alert analysis as outliers. The first was 
Matrix Run #25 Flown on October 1 7 th . It appears as though the timing of the scenario was off. 
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as ownship was on the runway when the alert was generated. The second outlier was Matrix Run 
#43 flown on October 19 th . There was no ADS-B or STIS-B traffic data for this run in the DAS 
files. Without this data, the run could not be analyzed. 


Scenario 4 


Table 27 presents alert data for the RSM Scenario 4 runs. Important variables for this scenario 
are ownship distance to threshold, separation distance, traffic distance to runway edge and traffic 
speed. As shown in Table 27, the average ownship distance to threshold at the time of the first 
alert was 1663.2 meters before the runway threshold. The alerts occurred while ownship’s 
distance to threshold ranged from 1342.2 to 1792.5 meters. The average separation distance at 
the time of the first alert was 3424 meters. The average traffic distance to runway edge at the 
time of first alert was 32 meters (away from runway). Traffic’s average speed at the time of first 
alert was 12.6 m/s or 25 kts, which did not vary much from run to run. 

Two runs, Matrix Run #26, and Matrix Run #66 of October 18 th and 20 lh respectively, were 
eliminated from this analysis as outliers. Both appear to be the result of bad scenario timing, as 
traffic was on the runway at the time of first alert for both runs. 

RSM False Alerts 

Not all false alerts were received during RIPS incursion runs. Every false alert captured in the 
DAS data has been noted, regardless of the type of test being performed. A detailed description 
of each false alert can be found Appendix H. RSM generated 4 false alerts throughout all of 
DFW testing. It appears as though all four were directly caused by ownship’s own STIS-B- 
generated position track. This track, referred to as “ownship’s ghost”, would tend to lag a few 
seconds behind ownship’s true position due to latency in the STIS-B data. When ownship 
decelerated, the distance between the ghost and ownship would begin to decrease. On a number 
of runs, this would cause an alert as there appeared to be a vehicle approaching ownship from 
behind at close range. Filtering out the STIS-B-based ID for ownship before processing STIS-B 
data would solve this problem. This was difficult to do during DFW testing, as ownship was 
given multiple IDs in the STIS-B data. 
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Table 20. Scenario 1 RSM Alert Summary 


Date 

Matrix 

# 

ISuM 

Alert 

On- 

Time 

Alert 

Late 

False 

Alert 

YES 

False 

Alert 

NO 

Notes 

16-Oct 

1 


XX 



XX 



9 


XX 



XX 



5 


XX 



XX 


17-Oct 

19 


XX 



XX 



23 


XX 



XX 


■IsSSBI 

27 






RSM did not alert, erroneous ADS-B data. 

18-Oct 

45 


XX 



XX 



37 


XX 



XX 


U2B2Z39 

41 


XX 



XX 


20-Oct 

63 


XX 



XX 



55 


XX 



XX 



59 


XX 



XX 



Table 21. Scenario 2 RSM Alert Summary 


Date 

Matrix 

# 


Alert 

On- 

Time 

Alert 

Late 

False 

Alert 

YES 

False 

Alert 

NO 

Notes 


10 


XX 



XX 



2 


XX 



XX 


17-Oct 

6 


XX 



XX 


17-Oct 

20 


XX 



XX 


17-Oct 

24 


XX 



XX 


18-Oct 

28 


XX 



XX 


18-Oct 

42 


XX 



XX 


18-Oct 

38 


XX 



XX 


19-Oct 

46 


XX 



XX 


20-Oct 

56 


XX 




RSM did not alert. ADS-B did not update 
for 30 sec. 

20-Oct 

64 


XX 



XX 
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Table 22. Scenario 3 RSM Alert Summary 


Date 

Matrix 

# 


Alert 

On- 

Time 

Alert 

Late 

False 

Alert 

YES 

False 

Alert 

NO 

Notes 


3 


XX 



XX 



11 


XX 



XX 


17-Oct 

7 




XX 


False alert on ID 84 (not RIPS Van - looks like 
ownship’s ghost) This occurred on approach, 
3.5 minutes before the actual incursion. 

17-Oct 

21 


XX 



XX 


17-Oct 

25 


XX 



XX 


17-Oct 

29 


XX 



XX 


18-Oct 

47 


XX 



XX 

Data gap. 

18-Oct 

39 


XX 



XX 

ADS-B data gap. 

19-Oct 

43 


XX 



XX 

No ADS-B data. 

20-Oct 

57 


XX 



XX 


20-Oct 

65 


XX 



XX 


20-Oct 

61 






RSM did not alert. No STIS-B data received. 


Table 23. Scenario 4 RSM Alert Summary 


Date 

Matrix 

# 


Alert 

On- 

Time 

Alert 

Late 

False 

Alert 

YES 

False 

Alert 

NO 

Notes 


4 


XX 



XX 


16-Oct 

8 


XX 



XX 


17-Oct 

12 


XX 



XX 


17-Oct 

22 


XX 



XX 


18-Oct 

26 


XX 



XX 


18-Oct 

30 






RSM did not alert, erroneous ADS-B data. 

18-Oct 

44 


XX 



XX 


18-Oct 

40 


XX 



XX 


19-Oct 

48 


XX 



XX 


20-Oct 

58 


XX 



XX 


20-Oct 

62 


XX 



XX 


20-Oct 

66 


XX 



XX 
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Table 24. Scenario 1 RSM Alerts 


Scenario 1 

Date 

UTC 

Matrix Run # 

Ownship Dist 
to T.H. (m) 

Traffic Dist. To 
H.L. (m) 

Traffic Dist. To 
RWY (m) 

16-Oct 

24955.5 

9 

-1686.1 

22.0 

-55.5 

17-Oct 

20628.3 

5 

-1426.2 

35.6 

-41.9 

17-Oct 

32229.1 

19 

-1739.9 

40.7 

-36.8 

18-Oct 

19079.3 

23 

-1630.3 

45.3 

-24.2 

18-Oct 

28680.3 

45 

-1922.9 

23.2 

-46.3 

18-Oct 

31388.3 

37 

-1842.5 

25.4 

-44.1 

20-Oct 

20960.8 

63 

-1848.8 

26.8 

-42.7 

20-Oct 

29978.5 

55 

-1839.6 

15.3 

-54.1 

20-Oct 

32730.8 

59 

-1846.5 

43.2 

-26.3 


AVERAGE 

-1753.6 

30.8 

-41.3 

STDEV 

153.6 

10.6 

10.8 

MINIMUM 

-1426.2 

15.3 

-24.2 

MAXIMUM 

-1922.9 

45.3 

-55.5 


Note - Distances to threshold, runway, and hold line have positive values for locations across these points and 
negative values for distances further away. 


Table 25. Scenario 2 RSM Alerts 


Scenario 2 

Date 

UTC 

Matrix Run # 

Ownship 
Speed (m/s) 

Separation Dist. 
(m) 

Traffic Dist. To 
H.L. (m) 

Traffic Dist. To 
RWY (m) 

16-Oct 

33849.2 

10 

22.6 

3222.0 

60.0 

-9.2 

17-Oct 

22953.3 

2 

35.8 

3172.8 

15.3 

-54.1 

17-Oct 

24304.2 

6 

40.0 

3101.5 

7.5 

-62.0 

17-Oct 

30972.1 

20 

37.8 

3184.8 

30.3 

-39.2 

17-Oct 

35896.9 

24 

50.3 

3021.2 

43.2 

-26.3 

18-Oct 

22761.8 

28 

36.2 

3164.3 

20.6 

-56.9 

18-Oct 

27462.4 

42 

43.6 

3066.6 

19.2 

-58.3 

18-Oct 

32516.4 

38 

39.7 

3086.4 

44.2 

-33.3 

19-Oct 

24934.6 

46 

35.6 

3151.8 

36.4 

-41.1 

20-Oct 

33874.6 

64 

36.0 

3183.3 

19.2 

-58.3 


AVERAGE 

37.8 

3135.5 

29.6 

-43.9 

STDEV 

7.0 

63.3 

16.2 

17.2 

MINIMUM 

22.6 

3021.2 

7.5 

-9.2 

MAXIMUM 

50.3 

3222.0 

60.0 

-62.0 
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Table 26. Scenario 3 RSM Alerts 


Scenario 3 

Date 

UTC 

Matrix Run # 

■I 

Ownship 
Dist. To 
RWY (m) 

T raffic 
Speed 
(m/s) 

Separation Dist. 
(m) 

16-Oct 

33494.0 

11 

46.1 

-31.4 

30.4 

2265.2 

16-Oct 

23018.7 

3 

14.3 

-63.2 

29.8 

2072.2 

17-Oct 

19566.1 

7 

22.3 

-52.4 

17.5 

3241.2 

17-Oct 

27968.8 

21 

58.3 

-19.2 

17.5 

3346.2 


34252.3 

29 

23.1 

-51.5 

13.4 

3402.8 


30251.0 

47 

24.3 

-45.2 

' 29.3 - 

2935.3 

18-Oct 

33017.9 

39 

12.4 

-57.1 

13.4 

3384.4 

20-Oct 

17482.9 

57 

16.6 

-51.9 

30.4 

3097.0 

20-Oct 

19932.1 

65 

35.1 

-34.4 

30.4 

3018.3 


AVERAGE 

28.0 

-45.1 

23.6 

2973.6 

STDEV 

15.5 

14.1 

7.8 

486.9 

MINIMUM 

12.4 

-19.2 

13.4 

2072.2 

MAXIMUM 

58.3 

-63.2 

30.4 

3402.8 


Table 27. Scenario 4 RSM Alerts 


Scenario 4 

Date 

UTC 

Matrix Run # 

Ownship 
Dist. To T.H. 

(m) 

Separation 
Dist. (m) 

%mmm 

T raffic 
Speed 
(m/s) 

16-Oct 

21422.4 

4 

-1585.6 

3261.0 

-30.0 

13.4 

16-Oct 

24193.0 

8 

-1342.2 

3006.5 

-23.6 

14.9 

17-Oct 

21476.4 

12 

-1691.1 

3363.3 

-42.2 

11.3 

17-Oct 

29451.5 

22 

-1792.5 

3465.5 

-37.9 

11.8 

18-Oct 

29302.3 

44 

-1718.6 

3528.2 

-31.5 

13.9 

18-Oct 

30900.4 

40 

-1757.7 

3562.4 

-38.6 

12.4 

19-Oct 

21487.2 

48 

-1772.9 

3577.8 

-28.6 

13.4 

20-Oct 

18467.3 

58 

-1715.0 

3517.5 

-36.5 

11.8 

20-Oct 

21834.3 

62 

-1593.0 

3400.0 

-19.3 

10.8 


AVERAGE 

-1663.2 

14.9 

-32.0 

12.6 

STDEV 

140.6 

182.8 

7.5 

1.3 

MINIMUM 

-1342.2 

3006.5 

-19.3 

10.8 

MAXIMUM 

-1792.5 

3577.8 

-42.2 

14.9 
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4.4.3 Ground-Based System Alerting Performance 

GBS had 1 1 missed alerts and 9 false alerts, with a total of 34 out of 47 runs resulting in proper 
alerts during DFW testing. The majority of the false alerts were caused by a false target, while 
others appeared to be the result of erroneous traffic altitude and speed data. 

4.4.3. 1 GBS Scenario Alert Plots 

Figures E-l through E-4 are plots of actual GBS runs, performed during testing at DFW. These 
plots represent typical GBS performance with good traffic data and proper scenario timing. 
Scenarios 1 and 2 were STIS-B only while scenarios 3 and 4 were ADS-B/STIS-B runs. 

Figure E-l is a plot of a Scenario 1 run, representing an Arrival/Taxi situation. As ownship gets 
within approximately 1300 meters of the runway threshold and traffic is on the runway, an RCA 
is issued. A few seconds later, ownship initiates a go-around at 277 ft altitude, safely clearing 
traffic and a potential collision. Traffic proceeds across the runway, ending the scenario. 

Figure E-2 is a plot of a Scenario 2 run. This scenario represents a Departure/Taxi situation. 
Ownship pulls onto the runway and proceeds to begin its take off roll. Once ownship has 
reached a minimum departure speed, traffic rolls across the hold line and continues toward the 
runway. With traffic on the runway, an RCA is generated with a vehicle separation distance of 
2616 m. Ownship is traveling at 1 17 kts at this point. Several seconds later, ownship initiates an 
aborted take off, slowing to taxi speed and safely pulling off the runway. 

Figure E-3 is a plot of a Scenario 3 run. This is very similar to Scenario 2, with ownship and 
traffic switching roles. Traffic pulls onto the runway and proceeds to begin its take off roll. 
Once traffic has reached a minimum take off speed, ownship begins to roll across the hold line. 
Ownship continues crossing onto the runway. An RCA is issued with a vehicle separation of 
2321 meters. Traffic is traveling at 64 kts at the time of alert. Ownship comes to a halt in 
response to the RCA, stopping a safe distance from the runway edge and avoiding a potential 
collision with the departing traffic. 

Figure E-4 is a plot of a Scenario 4 run. This scenario represents an Arrival/Departure situation. 
Ownship is preparing for an arrival. Traffic crosses the hold line and continues toward the 
runway. As ownship nears 1400 meters from the runway threshold, an RCA is issued. Traffic is 
on the runway traveling at 28 kts. Ownship initiates a go-around at 228 ft altitude, and safely 
avoids a potential collision with the departing traffic. 

4.4.3.2 GBS Alert Performance Summary 

GBS alerts are summarized in Tables 28 through 31, and alert data is presented in Tables 32 
through 35 and is sorted by scenario. A comprehensive summary of GBS alerting performance is 
found in Appendix F. 

It should be noted that GBS alerting performance varied throughout testing as a number of 
criteria thresholds were changed midway through the testing process. Originally, arrivals were 
not processed until they were within 0.5 nm of the runway threshold. This was extended to 1 nm. 
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In addition, Scenarios 1 and 2 were changed so that traffic would actually cross the runway, 
versus just crossing the hold line. GBS is designed to process safety logic on aircraft only when 
they have reached the runway or “runway corridor”. In the original scenario designs, the 
incursion vehicle stopped before it reached the runway corridor. As a result, no alerts were 
generated by GBS until the scenarios were changed to account for this. It appears as though the 
original criteria and scenario designs were major contributors to, if not the direct causes of the 
missed alerts in Scenario 1 for October 16 th and 17 th . 

Scenario 1 


Table 32 summarizes Scenario 1 alerting for GBS. Important variables for Scenario 1 are 
ownship distance to threshold and traffic distance to runway edge. The average ownship 
distance to threshold at the time of first alert was 1286.5 meters before the runway threshold. 
Ownship distance to threshold ranged from 1160.8 to 1546.9 meters. The average traffic 
distance over runway edge at time of first alert was 1 1.6 meters. Traffic was on the runway at the 
time of first GBS alert for all Scenario 1 runs. On one run, Matrix Run #63, October 20 th , GBS 
generated a false alert more than one minute before it should have alerted. This run was removed 
for the purpose of this analysis. 

Scenario 2 


Table 33 summarizes Scenario 2 GBS alert performance. Important variables for this scenario 
are ownship speed, separation distance and traffic distance to runway edge. The average 
ownship departure speed at the time of the first RCA was 39.9 m/s (78 kts). The highest 
recorded speed at the time of first alert was 59.8 m/s (117 kts). The average separation distance 
at the time of the first alert was 2638.8 meters. GBS alert thresholds are set such that distances 
of less than 3658 meters will trigger an alert for a DEPARTURE approaching a TAXI. The 
average traffic location was 22 meters over the runway edge. One run. Matrix Run #10, flown on 
October 16 th generated a false alert and was considered an outlier for this analysis. 

Scenario 3 


Table 34 summarizes GBS alert performance for the Scenario 3 runs. Important variables for 
this scenario are ownship distance to runway edge, traffic speed and separation distance. The 
average ownship distance to runway edge at the time of first alert was 19.3 meters. Ownship was 
on the runway at the time of all GBS alerts for Scenario 3. It should be noted that the GBS alerts 
use the aircraft centroid as an approximate reference point. This is approximately 24 m behind 
the nose. Therefore the alerts occurred with the GBS reference point nominally 5 m away from 
the runway edge. This is in accordance with the logic in the algorithms at the time of the test. 
The average traffic speed at the time of first alert was 29.8 m/s (58 kts), and the average 
separation distance was 2512.3 meters. As shown in Table 29 there were several runs where no 
alert was generated due to the fact that the aircraft stopped before the criteria for a GBS alert was 
satisfied. This occurred when one of the other alerting algorithms was being displayed in the 
cockpit. As soon as the pilot received one of those alerts the aircraft was immediately stopped. 
In runs 3, 7, 43, and 57 ownship stopped before GBS could alert. 
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Scenario 4 


Table 35 summarizes GBS alert performance for the Scenario 4 runs. Important variables for 
this scenario are ownship distance to threshold, separation distance, traffic distance to runway 
edge and traffic speed. At the time of the first alert, the average ownship distance to threshold 
was 1266.5 meters before the runway threshold. Values for this distance varied from 1098.3 to 
1403 meters. The average separation distance at the time of the first alert was 3133 meters. The 
average traffic distance to runway edge at the time of first alert was 23.4 meters. Traffic was on 
the runway at the time of the first GBS alert for all valid Scenario 4 runs. The average traffic 
speed at time of first alert was 17.7 m/s (34 kts). Two runs, Matrix Run #8, flown October 16 th 
and Matrix Run #62, flown October 20 th were both treated as outliers for Scenario 4 due to bad 
scenario timing and a false alert. 

GBS False Alerts 

GBS had 9 false alerts throughout DFW testing. Not all false alerts were received during RIPS 
incursion runs. Every false alert captured in the DAS data has been noted, regardless of the type 
of test being performed. GBS false alerts are summarized in detail in Appendix H. A few false 
alerts appeared as though they may have been caused by faulty altitude and heading data for 
target IDs, but the majority of the false alerts were caused by one apparent false target that was 
located well off the East side of Runway 35C. This target changed IDs a number of times 
throughout testing. It remained in the same location (1879.9, 1859.8) meters on the local 
coordinate system for the airport. This appears to be on or near Exit Q3.1 of Runway 35R. This 
target caused GBS to generate alerts for long time periods. On a few occasions, the alerts would 
start while ownship was miles out from the airport, and continue through an entire run into the 
next. These false alerts resulted in the erroneous reporting of a Runway Conflict Alert. 
Preliminary analysis indicates that these false alerts may have been related to a GBS 
STOPPED_TIMEOUT alert. The GBS STOPPED_TIMEOUT alert is designed to provide ATC 
with an alert to indicate that a target is stopped on the runway for an extended period of time. 
This is not a runway incursion alert. The RIPS avionics is designed for runway incursion 
alerting, not other types of alerting. For future implementations a review regarding which GBS 
alerts are transmitted to the RIPS avionics needs to be performed. In summary, there were no 
false alerts involving the aircraft and vehicle used in the RIPS testing. All were due to other 
targets received by the ground surveillance system. 
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Table 28. Scenario 1 GBS Alert Summary 


Date 

Matrix 

# 


Alert 

On- 

Time 

Alert 

Late 

False 

Alert 

YES 

False 

Alert 

NO 

Notes 

16-Oct 

1 






GBS did not alert on this run. 

16-Oct 

9 






GBS did not alert on this run. 

17-Oct 

5 






GBS did not alert on this run. 

17-Oct 

19 






GBS did not alert on this run. 

18-Oct 

23 


XX 



XX 


18-Oct 

U2JM 


XX 



XX 

ADS-B data gap. 

18-Oct 

45 


XX 



XX 


18-Oct 

37 


XX 



XX 


19-Oct 

41 


XX 



XX 


20-Oct 

63 




XX 


GBS falsely alerted throughout entire run. 
Note, the DAS data only displays one alert, 
cannot determine if the proper alert was 
generated and not recorded. 

20-Oct 

55 


XX 



XX 


20-Oct 

59 


XX 



XX 



Table 29. Scenario 2 GBS Alert Summary 


Date 

Matrix 

# 


Alert 

On- 

Time 

Alert 

Late 

False 

Alert 

YES 

False 

Alert 

NO 

Notes 

16-Oct 

10 




XX 


GBS alerted on ownship before the run was 
performed. Note, the DAS data only displays 
one alert, cannot determine if the proper alert 
was generated and not recorded. 


2 





XX 



6 






GBS did not alert on this run. 

17-Oct 

20 


XX 



XX 

ADS-B data gap. 


24 


XX 



XX 


Kiassa 

28 


XX 



XX 


18-Oct 

42 


XX 



XX 


18-Oct 

38 


XX 



XX 


19-Oct 

46 




XX 


GBS alerted constantly throughout entire run on 
a stationary traffic ID 207. Note, the DAS data 
only displays one alert, cannot determine if the 
proper alert was generated and not recorded. 

20-Oct 

56 


XX 



XX 

ADS-B data gap. 

20-Oct 

64 


XX 



XX 
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Table 30. Scenario 3 GBS Alert Summary 


Date 

Matrix 

# 

ISfiW 

Alert 

On- 

Time 

Alert 

Late 

False 

Alert 

YES 

False 

Alert 

NO 

Notes 

16-Oct 

3 






No alert - ownship stopped before alert criteria 
was met. 

16-Oct 

11 


XX 



XX 


17-Oct 

7 






No alert - ownship stopped before alert criteria 
was met. 

17-Oct 

21 


XX 



XX 


17-Oct 

25 


XX 



XX 


17-Oct 

29 






DAS data file stopped recording before end of 
run - it cannot be determined if GBS alerted. 

18-Oct 

47 


XX 



XX 

ADS-B data gap. 

18-Oct 

39 


XX 



XX 


19-Oct 

43 






No alert - ownship stopped before alert criteria 
was met. 

20-Oct 

57 






No alert - ownship stopped before alert criteria 
was met. 

20-Oct 

65 


XX 



XX 


20-Oct 

61 

■ 



XX 


GBS falsely alerted throughout entire run. 
Note, the DAS data only displays one alert, 
cannot determine if the proper alert was 
generated and not recorded. 


Table 31. Scenario 4 GBS Alert Summary 


Date 

Matrix 

# 


Alert 

On- 

Time 

Alert 

Late 

False 

Alert 

YES 

False 

Alert 

NO 

Notes 


4 






GBS did not alert on this run. 


8 


XX 



XX 


17-Oct 

12 


XX 



XX 



22 


XX 



XX 



26 


XX 



XX 


18-Oct 

30 


XX 



XX 

ADS-B data gap. 


44 


XX 



XX 



40 


XX 



XX 


19-Oct 

48 


XX 



XX 


20-Oct 

58 


XX 



XX 


20-Oct 

62 




XX 


GBS falsely alerted throughout entire run. The 
GBS alert type did switch when the proper alert 
should have been received, then switched 
back. 

20-Oct 

66 


XX 



XX 
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Table 32. Scenario 1 GBS Alerts 


Scenario 1 

Date 

UTC 

Matrix Run # 




18-Oct 

19085.2 

23 

-1246.5 

88.8 

20.3 

18-Oct 

20955.0 

27 

-1160.8 

77.6 

8.1 

18-Oct 

28691.1 

45 

-1195.7 

90.4 

20.9 


31395.2 

37 


79.0 

9.5 


18542.3 

41 

-1190.0 

77.6 

8.1 

20-Oct 

29987.4 

55 

-1274.5 

75.4 

5.9 

20-Oct 

32735.7 

59 

-1546.9 

77.6 

8.1 


AVERAGE 

-1286.5 

80.9 

11.6 

STDEV 

137.8 

6.0 

6.3 

MINIMUM 

-1160.8 

75.4 

5.9 

MAXIMUM 

-1546.9 

90.4 

20.9 


Note - Distances to threshold, runway, and hold line have positive values for locations across these points and 
negative values for distances further away. 


Table 33. Scenario 2 GBS Alerts 


Scenario 2 

Date 

UTC 

Matrix Run # 

Ownship 
Speed (m/s) 

Separation 
Dist. (m) 

KHBflB 


17-Oct 

22965.1 

2 

23.7 

2766.6 

91.8 

22.3 

17-Oct 

30984.8 

20 

31.3 

2697.3 

94.0 

24.5 

17-Oct 

35903.8 

24 

59.8 

2616.3 

94.0 

24.5 

18-Oct 

22774.5 

28 

30.1 

2700.0 

80.0 

5.4 

18-Oct 

27477.2 

42 

55.2 

2233.8 

90.0 

15.4 

18-Oct 

32526.2 

38 

37.9 

2682.7 

109.3 

30.8 

20-Oct 

24532.4 

56 

56.1 

2598.1 

97.8 

20.3 

20-Oct 

33886.4 

64 

25.2 

2815.5 

111.4 

32.9 


AVERAGE 

39.9 

2638.8 

96.0 

22.0 

STDEV 

14.8 

178.4 

10.2 

8.7 

MINIMUM 

23.7 

2233.8 

80.0 

5.4 

MAXIMUM 

59.8 

2815.5 

111.4 

32.9 
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Table 34. Scenario 3 GBS Alerts 


Scenario 3 

Date 

UTC 

Matrix Run # 


Ownship 
Dist. To 
RWY (m) 

T raffic 
Speed 
(m/s) 

Separation 
Dist. (m) 

16-Oct 

33514.7 

11 

94.7 

17.3 

27.8 

1727.7 

17-Oct 

27984.6 

21 

112.1 

34.6 

28.3 

3019.8 

17-Oct 

31367.3 

25 

94.7 

17.2 

26.8 

3065.4 

18-Oct 

30271.7 

47 

83.8 

14.3 

31.4 

2305.4 

18-Oct 

33047.4 

39 

81.6 

12.1 

31.9 

2634.1 

20-Oct 

19953.7 

65 

89.9 

20.4 

32.9 

2321.3 


AVERAGE 

92.8 

19.3 

29.8 

2512.3 

STDEV 

10.9 

8.0 

2.5 

504.6 

MINIMUM 

81.6 

12.1 

26.8 

1727.7 

MAXIMUM 

112.1 

34.6 

32.9 

3065.4 


Table 35. Scenario 4 GBS Alerts 



Date 

UTC 

Matrix Run # 

Ownship Dist. 
To T.H. (m) 

Separation 
Dist. (m) 


Traffic 

Speed 

(m/s) 

17-Oct 

21484.3 

12 

-1182.7 

2950.5 

20.1 

17.5 

17-Oct 

29460.3 

22 

-1216.7 

2989.2 

21.5 

17.5 

18-Oct 

18492.1 

26 

-1385.7 

3264.1 

12.2 

16.5 

18-Oct 

21666.3 

30 

-1313.4 

3211.0 

45.1 

21.6 

18-Oct 

29309.1 

44 

-1260.5 

3136.3 

26.8 

16.0 

18-Oct 

30910.3 

40 

-1098.3 

3044.8 

22.9 

20.1 

19-Oct 

21493.1 

48 

-1403.0 

3272.5 

19.3 

14.4 

20-Oct 

18475.2 

58 

-1183.3 

3084.5 

20.7 

18.0 

20-Oct 

30644.5 

66 

-1354.6 

3244.4 

21.5 

17.5 


AVERAGE 

-1266.5 

3133.0 

23.4 

17.7 

STDEV 

104.6 

122.1 

9.0 

2.1 

MINIMUM 

-1098.3 

2950.5 

12.2 

14.4 

MAXIMUM 

-1403.0 

3272.5 

45.1 
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4.4.4 Relative Comparison of Alert Performance 

4.4.4.1 Combined Alert Plots 

Figures G-l through G-4 are plots of actual RIPS runs conducted at DFW. Each plot displays the 
performance of all three systems, indicating ownship and traffic location at the time of the first 
alert for each individual system. All four scenarios are represented here, displaying typical 
performance of each system for every scenario. It should be noted that the performance of each 
system, as portrayed in these plots, may be affected when one of the other systems is the display 
system for a given run. Ownship response (i.e. rejected take off, go-around, emergency stop, etc) 
will be dictated by the alerts of the display system selected in the cockpit at that time of that run. 

Figure G-l is a plot of scenario 1. As traffic first crosses the hold line, a RIAAS RTA is 
generated followed immediately by an RSM alert. Traffic continues to taxi onto the runway. 
Just after traffic crosses the runway edge, a GBS alert is generated, followed a few moments later 
by a RIAAS RCA as traffic is approximately on the centerline of the runway. Ownship initiates 
a go-around in response to the RIAAS RCA, as RIAAS was the display system for this run. 

This plot gives a depiction of typical system performance for all three systems in scenario 1. 
Generally, the RIAAS RTA came a few seconds before the RSM alert. Usually eight to ten 
seconds later, the GBS alert was generated. Finally, approximately five seconds after the GBS 
alert, the RIAAS RCA was generated. This observed performance was consistent throughout 
testing at DFW. 

Figure G-2 is a plot of scenario 2. Ownship pulls onto the runway and proceeds to begin its take 
off roll. Traffic rolls across the hold line, resulting in a RIAAS RTA, followed immediately by a 
RIAAS RCA and an RSM alert. Traffic continues to cross the runway. Just as traffic passes the 
centerline of the runway, a GBS alert is generated. Ownship initiates a rejected take off in 
response to the RIAAS RCA, as RIAAS was the display system for this run. 

Figure G-2 shows typical performance in scenario 2 for the three systems throughout DFW 
testing. RIAAS generated an RTA, usually followed by an RCA one to two seconds later. The 
RSM alert would usually be generated at approximately the same time as the RIAAS RCA. 
Finally, GBS would generate an alert 10 to 15 seconds after the RIAAS RCA/RSM alerts. 

Figure G-3 is a plot of scenario 3. This scenario is very similar to scenario 2, with ownship and 
traffic reversing roles. Traffic pulls onto the runway and proceeds to begin its take off roll. 
Ownship rolls across the hold line, resulting in a RIAAS RTA. Ownship continues taxiing 
toward the runway, and a RIAAS RCA is generated, followed closely by an RSM alert. Ownship 
crosses onto the runway and moments later, a GBS alert is generated, allowing plenty of time for 
evasive action. GBS was the display system for this run. 

Figure G-3 displays typical performance of each system for scenario 3, as seen in DFW testing. 
A RIAAS RTA was generated, usually followed by a RIAAS RCA approximately 5 seconds 
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later. The RSM alert was generated usually within a second of the RIAAS RCA. The RSM alert 
was followed 10 to 20 seconds later by the GBS alert. 

Figure G-4 is a plot of scenario 4. This scenario represents an Arrival/Departure situation. As 
ownship is approaching for a landing, traffic rolls across the hold line toward the runway edge. 
A RIAAS RTA is generated soon after the hold line is violated, followed a few moments later by 
an RSM alert. Traffic pulls onto the runway and begins to accelerate into its take off roll. As 
traffic reaches the centerline a GBS alert is generated. Traffic continues to accelerate. As 
ownship gets closer to the runway threshold, the RIAAS RCA is generated. Ownship performs a 
go-around in response to the GBS alert, as GBS was the display system for this run. 

Figure G-4 displays typical performance of each system for scenario 4, as seen in DFW testing. 
A RIAAS RTA was generated, followed a few seconds later by the RSM alert. GBS would then 
alert 8 to 10 seconds after the RIAAS RTA and RSM alerts, followed by the RIAAS RCA 10 to 
15 seconds after that. 

4.4.4.2 Combined Alert Performance Summary 

The alerting performance of each system is broken down into two groups: ADS-B/STIS-B runs 
(both traffic data sources available) and STIS-B-only (ADS-B is turned off) runs. Total alert 
performance is also presented. In the STIS-B-only runs, GBS uses ASDE-3 and multilateration 
data only. Results for each system can be seen in Tables 36 through 38. 

For ADS-B/STIS-B runs, RIAAS alerted properly in 3 1 of 32 runs. The only missed alert was in 
a scenario 1, where the RIPS vehicle was moving but ADS-B updates showed no position change 
throughout the course of the run. RSM alerted properly on 29 of the 32 runs. One of the missed- 
alerts occurred in the same run as that mentioned for RIAAS, and was a result of erroneous ADS- 
B data. RSM also missed alerts in a scenario 2 and a scenario 4. Again with both of these 
scenarios, RSM did not alert because ADS-B position data indicated that the RIPS vehicle was 
not moving when in fact it was. 

For STIS-B only runs, RIAAS alerted properly in 13 of 15 runs. Missed alerts occurred in two of 
the four scenario 3 runs performed with RIAAS. In both missed-alert cases, no STIS-B data was 
transmitted. Because these were STIS-B only runs, no traffic data was available for alert 
processing. RSM alerted properly in 14 of 15 STIS-B-only runs. The only run in which RSM did 
not alert was one of the same scenario 3 runs in which RIAAS did not alert. Again, these were 
all a result of STIS-B-data not being available during STIS-B-only testing. 

GBS alerted properly in 34 of 47 runs. There were four missed alerts for scenario 1, two missed 
alerts for scenario 2, six missed alerts for scenario 3 and one missed alert for scenario 4. 
However almost all of the missed alerts were due to the original scenario design, where 
conditions to satisfy the GBS alerting criteria were not met. The first four runs for scenario 1 
were conducted prior to changing the alerting logic, thus no alerts were generated. Most of the 
missed alerts on scenario 3 were also due to the aircraft stopping prior to the alerting criteria 
being satisfied. Finally there were several runs where a false alert occurred, which may have 
prevented the system from recording the proper alert. 
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Overall, RIAAS alerted in 44 of 47 runs, RSM alerted in 43 of 47 runs, and GBS alerted in 34 of 
47 total runs. All missed alerts for both RIAAS and RSM were a direct result of erroneous or 
missing traffic data. Most of the missed alerts for GBS were related to the original alerting 
criteria, subsequently changed, and due to the conduct of some specific scenarios where the GBS 
alerting criteria were not satisfied. 
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Table 36. RIAAS Alert Summary By Data Source 



ADS-B/STIS-B 

STIS-B Only 

All Runs 

Scenario 

# Runs 

# Alerts 

# Runs 

# Alerts 

# Runs 

# Alerts 

1 

8 

7 

4 

4 

12 

11 

2 

8 

8 

3 

3 

11 

11 

3 

8 

8 

4 

2 

12 

10 

4 

8 

8 

4 

4 

12 

12 

Total 

32 

31 

15 

13 

47 

44 


Table 37. RSM Alert Summary By Data Source 



ADS-B/STIS-B 

STIS-B Only 

All Runs 

Scenario 

# Runs 

# Alerts 

# Runs 

# Alerts 

# Runs 

# Alerts 

1 

8 

7 

4 

4 

12 

11 

2 

8 

7 

3 

3 

11 

10 

3 

8 

8 

4 

3 

12 

11 

4 

8 

7 

4 

4 

12 

11 

Total 

32 

29 

15 

14 

47 

43 


Table 38. GBS Alert Summary By Data Source 



ADS-B/STIS-B 

STIS-B Only 

All Runs 

Scenario 

# Runs 

# Alerts 

# Runs 

# Alerts 

# Runs 

# Alerts 

1 

8 

5 

4 

3 

12 

8 

2 

8 

7 

3 

2 

11 

9 

3 

8 

5 

4 

1 

12 

6 

4 

8 

7 

4 

4 

12 

11 

Total 

32 

24 

15 

10 

47 

34 
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5.0 SUMMARY AND CONCLUSIONS 

The primary conclusion of this report is that the three types of approaches to generating runway 
incursion alerts in the cockpit demonstrated feasibility during the DFW RIPS testing. This 
included two different aircraft-based alerting systems and a ground-based system. Out of the 47 
test runs, RIAAS provided alerts on 44, RSM on 43, and GBS on 34. All of the missed alerts by 
RIAAS and RSM were a direct result of erroneous or missing traffic data. Most of the missed 
alerts for GBS were related to the original alerting criteria, subsequently changed, and due to the 
conduct of some specific scenarios where the GBS alerting criteria were not satisfied. In these 
instances the relative locations of the aircraft and test vehicle did not meet the GBS criteria for 
alert. RIAAS generated 2 false alerts during the testing, both the result of erroneous traffic data. 
RSM generated 4 false alerts, which were the result of the ownship-generated STIS-B traffic 
reports. GBS generated 9 false alerts during the testing, most of which were due to an apparent 
false ASDE-3 target located off the runway. The testing showed that the pilot could safely take 
evasive action (i.e., go-around, rejected take off, stop taxi) when the alerts normally occurred on 
all three systems for the four incursion scenarios tested. However, for the scenarios involving 
violation of hold lines, the GBS alerts occurred significantly later than did the aircraft-based 
systems. In those two scenarios (1 and 3) the GBS alerts did not occur until the vehicle/aircraft 
was on the runway. The two aircraft-based systems alerted well before the vehicle and aircraft 
reached the runway. 

Regarding the integration of the supporting airborne and ground systems, the test results indicate 
that the basic system architecture demonstrated will support both aircraft-based and ground- 
based incursion alerting. One obvious conclusion is that alert logic performance is very 
dependent on the performance of the traffic and ownship position information. This information 
must be reliable, timely and accurate to ensure optimum runway incursion alerting performance. 
The NASA B757 airborne systems demonstrated excellent performance with respect to ownship 
information. However, there were a number of issues identified regarding the generation and 
processing of traffic information using STIS-B and ADS-B. Missing or erroneous STIS-B and 
ADS-B data resulted in a number of missed, late, and false alerts. The prototype nature of the 
systems involved probably played a significant role in the availability and integrity of the traffic 
data. One specific conclusion regarding traffic information is that STIS-B information had 
significantly longer latency than did ADS-B. This translates directly into delayed alerting on 
targets using position reports from STIS-B. ADS-B position reports were also significantly more 
accurate than STIS-B. 

The testing showed that aircraft-based alerting has demonstrated several key advantages over 
having ground based alerts provided via data link, including: 

• Shorter time delay between the time the alerts are generated and when they are 
annunciated to the flight crew. 

• More timely alert generation. One reason for this is the capability to use ownship 
position data to accurately determine the ownship nose location. This provides a means 
to very accurately determine when ownship has violated a hold line on entering a runway. 
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A similar computation can be made for the tail location to determine when an aircraft has 
failed to clear the hold line on exiting a runway. 

• Ground infrastructure is not required when aircraft are equipped with ADS-B. 

Aircraft-based alerts provided to the flight crew will in some cases occur in advance of ground- 
based alerts provided to ATC. For example, in the case where ownship violates the hold line, an 
aircraft-based alert can occur sooner than the ground-based alerts due to the ability to accurately 
determine nose position. There is a safety benefit to alerting the flight crew as soon as the 
aircraft has crossed the hold line. This may present an issue regarding the difference in timing 
for the two alerting systems. The compatibility of aircraft-based alerts reported to the flight crew 
and ground-based alerts reported to ATC needs further investigation. 

One of the systems, RIAAS, demonstrated a two-stage alerting concept, which includes a Traffic 
Alert and a higher priority Conflict Alert. The other two, RSM and GBS, provided a single 
conflict alert. The intent of the two stage alerting is to provide advanced warning to the pilot of a 
pending conflict. When the Traffic Alert occurs, the flight crew has the option of either 
continuing the procedure or taking evasive action. The test results indicated that evasive action 
could be taken following the Conflict Alert, maintaining safe separation. For three of the 
scenarios tested there was sufficient time (10-20 seconds) between the two RIAAS alerts for the 
pilot to determine the best course of action. For the rejected take off scenario (2) there was 
minimal time between the two alerts, and in some instances only the Conflict Alert was 
generated. Further simulation and testing is required to validate and optimize the two-stage 
alerting approach. 

Analysis of the test results yielded several recommendations regarding the supporting 
infrastructure and the alerting systems, including: 

• Further development of ground and avionics systems should include enhancement of 
availability and integrity of ADS-B and STIS-B traffic information. The ground system 
should provide integrity monitoring of surveillance data prior to STIS-B transmission. 
The probability of transmitting false target data needs to be extremely low. STIS-B 
should transmit a parameter equivalent to the ADS-B Navigation Uncertainty (NUC). 
This will indicate the accuracy of the surveillance information. The latency in the STIS-B 
transmissions should also be minimized to reduce alert delays. 

• A reference point correction for the ADS-B target should be performed. It is 
recommended that the ADS-B MASPS (Minimum Aviation System Performance 
Standard) [5] be amended to include a requirement that the reported position is referenced 
to a standard location on the aircraft. If the position is provided to a known location then 
the alerting systems can apply the correction to other critical aircraft points of reference 
(i.e., nose, tail). 

• The ground system should provide STIS-B position reports that are corrected to a 
reference point, such as the nose or centroid of the aircraft. The ground system has 
knowledge of the surveillance sensor(s) used to determine the fused position. Each 
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sensor can use a different reference point. For instance, ASDE-3 position is referenced to 
the target centroid and multilateration position is referenced to the transponder antenna(s) 
location. The avionics does not have the knowledge of which sensor is used to compute 
the ground system derived traffic reports. 

• Careful review regarding which GBS alerts are transmitted to the RIPS avionics needs to 
be performed. RIPS is designed to provide runway incursion alerting. Other types of 
GBS alerts may not be appropriate for presentation to the flight crews. 

• Aircraft-based incursion alerting systems should incorporate some level of integrity 
checking on traffic information to minimize missed and false alerts. 
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APPENDIX A - RIAAS ALERT PLOTS 



Scenario 1 - Matrix 55 - October 20, 2000 
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Figure A-l 






Scenario 2 - Matrix 64 - October 20, 2000 



X Location (m) 




Scenario 3 - Matrix 57 - October 20, 2000 



X Location (m) 




Scenario 4 - Matrix 8 - October 16, 2000 



Figure A-4 




APPENDIX B - RIAAS ALERT SUMMARY 



Table B-l - RIAAS Alert Si 




























































































































































































Table B-l - RIAAS Alert Summary Continued 
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Table B-l - RIAAS Alert Summary Continued 

















































































































































Table B-l - RIAAS Alert Summary Continued 
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Table B-l - RIAAS Alert Summary Continued 


































































































































































Table B-l - RIAAS Alert Summary Continued 
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APPENDIX C - RSM ALERT PLOTS 



Scenario 1 - Matrix 63 - October 20, 2000 



Figure C-l 




Scenario 2 - Matrix 38 - October 18, 2000 




Scenario 3 - Matrix 65 - October 20, 2000 



X Location (m) 



Scenario 4 - Matrix 58 - October 20, 2000 



Figure C-4 



APPENDIX D - RSM ALERT SUMMARY 



Table D-l — RSM Alert Summary 



















































































































































































Table D-l - RSM Alert Summary Continued 













































































































































































































Table D-l - RSM Alert Summary Continued 


General Notes 


No traffic ADS-B data, and no traffic STIS-B data 

updates anywhere near time of alert (based on DAS 

data). Do not know source of data that RSM was 
alerting on. 
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APPENDIX E - GBS ALERT PLOTS 






Scenario 2 - Matrix 24 - October 17, 2000 



Figure E-2 







Scenario 4 - Matrix 48 - October 19, 2000 



Figure E-4 





APPENDIX F - GBS ALERT SUMMARY 



Table F-l - GBS Alert Summary 




g <| S3A 
(uu) uoflejedss 

(S/UU) p88dS 

( 6 aa) 6 u|pe 8 H 
(uu) Ae/wuny 01 ^sjo 
(uu) th oi }S\Q 

(uu) 'H'loi isj a 

(s/uu) pssds 

( 6 sq) 6 u|pe 8 H 

(uu) AeMuny 01 'jsjQ 

(uu) th °1 'l s !a 

(uu) 'H'loi \s\q 

sdAj psiv S 90 

psAeidsjQ uusjsAs 

« gl a-sus/a-sav 
q h g-sus 

# uny xu^ei/M 

# pjep ;u|B!|j 

#w 6 ||d 

0UBU80S 
(spuooss) P8|V 


isjjd p oin sva 

8}BQ 


i § 


to I a> 



■c -C 

0 _a) 

CO CO 



■g "a 

TD TD 

c/3 c/D 

m cq 

o o 






x 









CM 























































































































































Table F-l - GBS Alert Summary Continued 











































































































































































































































































































APPENDIX G - COMBINED ALERT PLOTS 



Scenario 1 - Matrix #55 October 20, 2000 



250 3f # Location (m) 450 






Scenario 2 - Matrix #64 - October 20, 2000 
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Figure G-2 
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Scenario 4 - Matrix #48 - October 19, 2000 





APPENDIX H - FALSE ALERTS 



Table H-l RIAAS False Alerts 
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Table H-2 - RSM False Alerts 


































































Table H- 3 - GBS False Alerts 





























































Table H-3 continued - GBS False Alerts 
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APENDIX I - ACRONYMS 



ADS-B 

Automatic Dependent Surveillance - Broadcast 

AGL 

Above Ground Level 

AMASS 

Airport Movement Area Safety System 

ARIES 

Airborne Research Integrated Experiment System 

ARTS 

Automated Radar Tracking System 

ASDE-3 

Airport Surface Detection Equipment radar 

ASR-9 

Airport Surveillance Radar 

ATC 

Air Traffic Control (Air Traffic Controller) 

ATIDS 

Airport Traffic Identification System 

CDTI 

Cockpit Display of Traffic Information 

CNS 

Communications Navigation and Surveillance 

CPDLC 

Controller-Pilot Data Link Communications 

DAS 

Data Acquisition System 

DFW 

Dallas-Fort Worth International Airport 

DGPS 

Differential Global Positioning System 

FAA 

Federal Aviation Administration 

FIS-B 

Flight Information Services - Broadcast 

GBS 

Ground-Based Alerting System 

GNSS 

Global Navigation Satellite Systems 

GPS 

Global Positioning System 

HSAFT 

Hold Short Advisory Landing Technology 

HUD 

Heads-Up Display 

IDS-RSM 

Integrated Display System - Runway Safety Monitor 

INS 

Inertial Navigation System 

FAAS 

Local Area Augmentation System 

NASA 

National Aeronautics and Space Administration 

ND 

Navigation Display 

NUC 

Navigation Uncertainty 

OD 

Operational Deviations 

OE 

Operational Errors 

PD 

Pilot Deviations 

PVT 

Position, Velocity and Time 

RCA 

Runway Conflict Alert 

RIAAS 

Runway Incursion Advisory and Alerting System 

RIPS 

Runway Incursion Prevention System 

RSM 

Runway Safety Monitor 

RTA 

Runway Traffic Alert 

RTO 

Rejected Take off 

SA 

Situational Awareness 

SGI 

Silicon Graphics Incorporated 

SSR 

Secondary Surveillance Radar 

STIS-B 

Surface Traffic Information Services - Broadcast 

TCAS 

Traffic Alerting and Collision Avoidance System 
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UAT 

UTC 

MM) 

WAAS 


Universal Access Transceiver 
Universal Time Constant 
Vehicle / Pedestrian Deviations 
Wide Area Augmentation System 
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